■0f 


\ 


AD-753  125 


REPRESENTATION  AND  ESTIMATION  OF  CYCLO¬ 
STATIONARY  PROCESSES  ! 


William  Allen  Gardner 

I 

Massachusetts  Uriiversity 


<*■ 


Prepared  for: 

i 

Air  Force  Office  of  Scientific  Research 
August  1972  1 


J 

S 


DISTRIBUTED  BY: 


I 


National  Technical  Information  Service 
U.  S.  DEPARTMENT  OF  COMMERCE 

5285  Port  Royal  Road,  Springfield  Va.  22151 


1*0 

“  01 

CO 

B»o 

N 

0  a 


of"*®* 


V 

V?. 


1 863' 


AFOSR  -  TR*72*2S7ft 


GfujineeliHC  Research  l Institute 

UNIVERSITY  OF  MASSACHUSETTS 
AMHERST,  MASSACHUSETTS 


crxrDv 


PEC  26  1972 

SSEOTJTSli 

Qi 


Approved  for  publio  release^ 
distribution  unlimited. 


T 


REPRESENTATION  AND  ESTIMATION  OF 
CYCLOSTAT IONAKY  PROCESSES 
by 

William  Allen  Gardner 

M.S.,  Stanford  University 
Ph.D.,  University  of  Massachusetts 

y  .  1  -  •  j) 

u  «•  1  >  I 

Amherst,  Massachusetts 
August  1972 


D  D  C 

BSEDM2 

DEC  26  1972 

IDIOTS 


/ 


Security  CaMifurttow 


DOCUMENT  CONTROL  DATA  -RAD 


i  ORIGINATING  ACTiviTT  (Corporal*  autfiotj 

2#,  REPORT  SECURITY  CLASSIFICATION 

Engineering  Research  Institute 

UNCLASSIFIED 

University  of  Massachusetts 

Amherst,  Massachusetts 

2b.  CROUP 

3  REPORT  TITLE 

! 

REPRESENTATION  AND  ESTIMATION  OF  CYCLOSTATIONARY  PROCESSES 

4.  DESCRIPTIVE  NOTES  (Typ»  ot  ttpctl  Ami  Incltulr. 

Scientific  Interim 

s  autnodiSi  (Flrtl  n.m.,  mlddlt  Iniltml.  iMliwmj 


William  A.  Gardner 


•  REPORT  DATE 

August  1972 

•*.  CONTRACT  OR 

GRANT  NO 

AFOSR  71-2111 

6.  PROJCC  T  NO- 

9769 

c. 

61102F 

\ 

d. 

681304 

7m.  TOT41  HO.  OF  PACES 

353 


M.  ORIGINATOR**  REPORT  NUMBCM |S> 


#5.  OTHEH  REPORT  NO(3)  (Any  olhmr  nizmbmta  Ihmt  aamy  be  maaifned  I 
ihlm  re  pc  ft)  ! 

yrw  ,  TD  «  7  9  -  9  q  7  g 
f.i  JOi\  1 11  *  ^o*o 


10  DISTRIBUTION  STATEMENT 


Appi  ">ved  for  public  release;  distribution  unlimited. 


M  SUPPLEMENT ARV  NOTES 


TECH.,  OTHER 


»*.  »PON»UHING  MJLITl.RV  AC  Tl  VI  ,  V 

Air  Force  Office  of  Scientific  Research/NM 
1400  Wilson  Boulevard 
Arlington,  Virginia  22209 


•3  ABSTRACT 


Random  signal  processes  which  have  been  subjected  to  some  form  of  repetitive  operation 
such  as  sampling,  scanning  or  multiplexing  will  usually  exhibit  statistical  properties 
which  vary  periodically  with  time.  In  many  cases,  the  repetitive  operation  is  introducec 
intentionally  to  put  the  signal  in  a  format  which  is  easily  manipulated  and  which 
preserves  the  time-position  integrity  of  the  events  which  the  signal  is  representing. 
Familiar  examples  are  radar  antenna  scanning  patterns,  raster  formats  for  scanning 
video  fields,  synchronous  miltiplexing  schemes,  and  synchronizing  and  framing  tech¬ 
niques  employed  in  data  transmission.  In  fact,  in  all  forms  of  data  transmission,  it 
seems  that  some  form  of  periodicity  is  imposed  on  the  signal  format. 

Random  processes  with  statistical  properties  that  vary  periodically  with  time  are 
encountered  frequently,  not  only  in  electrical  communication  systems,  but  in  biological 
systems,  chemical  processes,  and  studies  concerned  with  meteorology,  ecology,  and 
other  physical  and  natural  sciences.  . 

Systems  analysts  have  tended^  f  or  the  most  part,  to  treat  these  "cyclostationary1' 
processes  as  though  they  were  stationary.  This  is  done  simply  by  averaging  the 
statistical  parameters  (mean,  variance,  etc.)  over  one  cycle.  This  averaging  is 
equivalent  to  modelling  the  time-reference  or  phase  of  the  process  as  a  random  variable 
uniformly  distributed  over  one  cycle.  Thin  type  of  analysis  is  appropriate  in  situations 
where  the  process  is  not  observed  in  synchronism  with  its  periodic  structure.  However, 
in  a  receiver  that  is  intended  for  a  cyclostationary  signal,  there  is  usually  provided 
a  great  deal  of  inf ormation--in  the  form  of  a  synchronizing  pulse-stream  or  a  sinusoidal 
timing  signal--about  the  exact  phase  of  the  signal  format.  Most  systems  are,  in  fact, 
inoperative  without  this  information. 
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ABSTRACT 

/ 

V1 

Random  signal  processes  which  have  been  subjected  to  some  form  of 
repetitive  operation  such  as  sampling,  scanning  or  multiplexing  will 
usually  exhibit  statistical  properties  which  vary  periodically  with  time. 

In  many  cases,  the  repetitive  operation  is  introduced  intentionally  to 
put  the  signal  in  a  format  which  is  easily  manipulated  and  which  preserves 
the  time-position  integrity  of  the  events  which  the  signal  is  representing. 
Familiar  examples  are  radar  antenna  scanning  patterns,  raster  formats  for 
scanning  video  fields,  synchronous  multiplexing  schemes,  and  synchronizing 
and  framing  techniques  employed  in  data  transmission.  In  fact,  in  all 
forms  of  data  transmission,  it  seems  that  some  form  of  periodicity  is 
imposed  on  the  signal  format. 

Random  processes  with  statistical  properties  that  vary  periodically 
with  time  are  encountered  frequently,  not  only  in  electrical  communication 
systems,  but  in  biological  systems,  chemical  processes,  and  studies 
concerned  with  meteorology,  ecology,  and  other  physical  and  natural  sciences. 

Systems  analysts  have  tended,  for  the  most  part,  to  treat  these 
"cyclostationary"  processes  as  though  they  were  stationary.  This  is  done 
simply  by  averaging  the  statistical  parameters  (mean,  variance,  etc.)  over 
one  cycle.  This  averaging  is  equivalent  to  modelling  the  time-reference 
or  phase  of  the  process  as  a  random  variable  uniformly  distributed  over 
one  cycle.  This  type  of  analysis  is  appropriate  in  situations  where  the 
process  is  not  observed  in  synchronism  with  its  periodic  structure. 

However,  in  a  receiver  that  is  intended  for  a  cyclostationary  signal, 
there  is  usually  provided  a  great  deal  of  information- -in  the  form  of  a 
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synchronizing  pulse-stream  or  a  sinusoidal  timing  signal— about  the  exact 
phase  of  the  signal  format.  Most  systems  are,  in  fact,  inoperative 
without  this  information. 

I 

The  first  chapter  of  this  dissertation -is  introductory  and  features 
a  detailed  historical  account  of  the  meager  development  and  application 
of  the  theory— still  in  its  infancy— of  cyclostationary  processes  (as  it 
appears  in  the  engineering  literature). 

The  second  chapter  is  an  extensive  treatment  of  the  topics  of 
transformation,  generation,  and  modelling  of  cyclostationary  processes;' 
and,  among  other  things,  serves  to  introduce  a  large  number  of  models 
for  cyclostationary  processes.  These  models  are  used  throughout  the 
dissertation  for  illustrating  various  theoretical  results. 

The  third  chapter  is  an  in-depth  treatment  of  series  representations 
for  cyclostationary  processes,  and  their  autocorrelation  unctions,  and 
other  periodic  kernels.  These  representations  are  applied  to  the  problems 
of  analysing  cyclostationary  processes,  solving  linear  integral  equations 
with  periodic  kernels,  realizing  periodically  time-varying  linear  systems, 
and  defining  a  generalized  Fourier  transform  for  cyclostationary  (and 
stationary)  processes. 

The  fourth  chapter  addresses  itself  to  the  problem  of  least-mean- 
squared-error  linear  estimation  (optimum  filtering)  of  cyclostationary 
processes,  and  employs  the  representations  of  Chapter  III  to  obtain  solutions, 
and  the  models  of  Chapter  II  to  illustrate  these  solutions.  Previous 
analyses  of  optimum  filtering  operations  have  assumed  the  stationary  model  for 
these  processes  and  result  in  time-invariant  filters.  One  of  the  major 
results  in  this  thesis  is  the  demonstration  of  the  improvement  in 


performance  that  can  be  obtained  by  recognizing  that— by  virtue  of 
timing  information  at  the  receiver—the  received  process  is  actually 
cyclostationary  and  the  optimum  filter  is  a  periodically  time-varying 
system.  Numerous  illustrative  examples  including  amplitude-modulation 
frequency-shift-keying,  pulse-amplitude-modulation,  frequency-division 
multiplexing,  and  time-division-multiplexing  are  worked  out  in  detail 
and  include  realizations  of  the  optimum  time-varying  filters. 
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CHAPTER  I 


INTRODUCTION 

1.  Motivation  and  Brief  Description  ^ 

Random  signal  processes  which  have  been  subjected  to  some  form  of 
repetitive  operation  such  as  sampling  or  scanning  will  usually  exhibit 
statistical  properties  which  vary  periodically  with  time.  In  many  cases, 
the  repetitive  operation  is  introduced  intentionally  to  put  the  signal 
in  a  format  which  is  easily  manipulated  and  which  preserves  the  time- 
position  integrity  of  the  events  which  the  signal  is  representing. 
Familiar  examples  are  radar  antenna  scanning  patterns,  raster  formats 
for  scanning  video  fields,  synchronous  multiplexing  schemes,  and 
synchronizing  and  framing  techniques  employed  in  data  transmission.  In 
fact,  in  all  forms  of  data  transmission,  it  seems  that  some  form  of 
periodicity  is  imposed  on  the  signal  format. 

Random  processes  with  statistical  properties  that  vary  periodically 
with  time  are  encountered  frequently,  not  only  in  electrical 
communication  systems,  but  in  studies  concerned  with  biological  systems, 
meteorology,  chemical  processes,  and  ecological  systems. 

One  process  which  exhibits  statistically  periodic  characteristics 
is  a  synchronously  timed  pulse -sequence  as  used  in  digital  data- 
transmission.  Bennett  [1]  recognized  this  as  a  special  case  of  a  broader 
class  of  processes  which  he  termed  cyclostationary  [2].  Other  authors 
[3 j - [9]  have  used  the  terms  "periodically  stationary",  "periodic¬ 
stationary",  "periodically  correlated",  "periodic-nonstationary"  and 
"periodically  nonstationary"  to  denote  this  same  class. 
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Systems  analysts  have  tended,  for  the  most  part,  to, treat  these 
processes  statistically  as  though  they  were  stationary  processes.  This 

i 

is  done  simply  by  averaging  the  statistical  parameters  (mean,  covariance, 
etc.)  over  one  period.  This  averaging,  is  equivalent  to  assuming  that 
the  time -reference,  or  phase,  of  the  repetitive  operation  is  completely 

i  I  ,  . 

indeterminate;  i.e.,  the  phase  is  a  randqm  variable,  uniformly  distributed 
oyer  one  period.  This  type  of  analysis  is  appropriate  in  situations 

i  1 

where  the  process  is  not  observed  (or  measured)  in  synchronism  with  its 

i  i  ■' 

periodic  structure.  For  example,  a  cyclostationary  process  may  be  an 
interference  in  a  signal -transmission-channel  whose  receiver  has  no 

i  * 

I 

knowledge  ofi  the  phase  of  the  interfering  process.  In  such  a  situation, 
the  concepts  used  with  stationary  processes,  such  as  power  spectral 

density,  can  be  valuable  dn  evaluating  system  performance..  However,  in 

!  .  • 

a  receiver  that  is  intended  for  a  cyclostationary  process,  there  is 

usually  provided  a  great  deal  of  information- -in  the ; form  of  a 

1  1  1  : 
synchronizing  pulse-stream  or  a  sinusoidal  timing  signal --about  the 

V  '  I 

exact  phase  of  the  signal  format.  Most  systems  are,  in  fact,  inoperative 

i  i  1 

without  this  information.  i  ,  1 


Previous  analyses  of  optimum-reqeiver  filtering  operations  have 
assumed  the . stationary  model  for  these  processes  and--for  least-mean- 

squared-error  reconstruction  of  the  signal--result  in  time-invariant 

\  '  • 

filters.-  One  of  the  major  results  in  this  thesis  is  the  demonstration 

of  the  improvement  in  performance  that  can  be  obtained  by  recbgnizipg 
that--by  virtue  of  timing  information  at  the  receivcr-,-the  received 
process  is  actually  cyclostationaiy  and  the  optimum  receiver- filter  is 
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a  periodically  time-varying  system.  Numerous  illustrative  examples 
including  amplitude -modulation,  frequency-shift -keying,  pulse-amplitude- 
modulation,  frequency-division-multiplexing,  and  time-division- 
multiplexing  are  worked  out  in  detail  and  include  realizations  of  the 
optimum  time -varying  filters. 

The  many  examples  presented  not  only  demonstrate  improvement,  but 
also  illustrate  the  general  methods  which  are  developed  for  representing 
cyclostationary  processes  and  solving  for  and  synthesizing  optimum 
periodic  filters.  These  methods  are  embodied  in  a  general  approach 
which  applies  not  only  to  the  problems  of  representation  and  estimation 
of  cyclostationary  processes,  but  more  extensively  to  the  solution  of 
linear  integral  equations  with  periodic  kernels  and  the  synthesis  of 
periodic  linear  systems. 

The  remainder  of  this  introductory  chapter  serves  to  define  cyclo- 
stationarity,  and  to  illustrate  same  with  several  examples  of  cyclo¬ 
stationary  signals  which  are  commonly  employed  in  communication  systems, 
and  to  give  a  historical  account  of  the  development  and  application  of 
the  theory  of  cyclostationary  processes  (as  it  appears  in  the  engineering 
literature) . 

The  second  chapter  is  an  extensive  treatment  of  the  topics  of 
transformation,  generation,  and  modeling  of  cyclostationary  processes; 
and,  among  other  things,  serves  to  introduce  a  large  number  of  classes, 
or  types,  of  models  for  cyclostationary  processes.  These  models  are 
used  throughout  the  dissertation  for  illustrating  various  theoretical 
results. 

The  third  chapter  is  an  in-depth  treatment  of  series 
representations  for  cyclostationary  processes,  and  their  autocorrelation 


functions,  and  other  periodic  kernels;  and  includes  brief  discussions 
of  their  application  to  the  solution  of  integral  equations,  and  to  the 
reali2  ,„ion  of  periodically  time-varying  systems. 

The  fourth  chapter  addresses  itself  to  the  problem  of  least-mean- 
square  linear  estimation  of  cyclostationary  processes  (in  noise),  and 

m 

employs  the  representations  of  Chapter  III  to  obtain  solutions,  and  the 
models  of  Chapter  II  to  illustrate  these  solutions. 

The  fifth  chapter  is  a  detailed  summary  of  the  contents  of  Chapters 
II,  III,  IV  and  includes  recommendations  for  further  research. 

As  a  convenience  for  the  reader,  this  final  chapter  serves  as  a 
combination  index,  outline  and  summary.  It  is  hoped  that  this  detailed 
summary  will  help  to  offset  the  length  of  Chapters  II,  III,  IV. 
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2.  Definition  of  Cyclostationarity 


We  begin  with  definitions  for  two  types  of  stationarity: 


DEFINITION:  A  random  process  (x(t);  t  e  J}  is  stationary-of-order-two 
(S(2))  if  and  only  if: 

i)  the  probability  density  functions (PDF's)  for  the  random  variables 
(x(t)}  are  identical  for  all  t  e  J: 


px(t)(,)  =  px(0)(,) 


¥•  t  e  J, 


ii)  for  every  t,s  e  J  the  joint  PDF  for  the  random  variables  x(t)  and 
x(s)  depends  only  on  the  difference  of  indices  t-s: 


px(t)x(s)(*’’J  =  px(t-s)x(0)(,,,)  *  t,s  e  J’ 

If  x  is  a  continuous-time  random  process,  then  the  index  set  J 
is  the  set  of  real  numbers  R,  and  if  x  is  a  random  sequence1  then  J  is 
the  set  of  integers  I. 


1.  We  are  using  an  extended  definition  of  a  random  sequence;  strictly 
speaking,  the  domain  of  a  sequence  is  the  natural  numbers  (positive 
integers).  Notice  also  that  if  x  is  a  random  sequence,  then  the  PDF's 
Px(0)^*  px(t)x(0)  are  comPosec*  one"  an<*  two-dimensional  impulse 

functions . 
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In  the  above  definition,  E{-}  denotes  statistical  expectation, 
and  *  denotes  complex  conjugation.  In  the  interest  of  notational 
economy,  we  will  (with  some  abuse  of  notation)  identify  the  auto¬ 
correlation  function  for  a  WSS  process  x  as  kxx(t-s)  =  kxx(t-s,0). 

Notice  that  stationary-of-order-two  processes  compose  a  subclass 
of  the  class  of  processes  which  are  wide-sense  stationary,  and  both  of 
these  classes  are  characterized  by  the  invariance  of  certain  statistical 
parameters  to  arbitrary  shifts  of  the  indexing  variable.  We  now  define 
two  superclasses  which  are  characterized  by  the  invariance  of  certain 
statistical  parameters  to  shifts  of  the  indexing  variable  by  multiples 
of  a  basic  period.  To  emphasize  the  cyclic  character  of  these  processes, 
the  term  cyclostationary  will  be  used: 

DEFINITION:  A  random  process  (x(t);  t  c  J)  is  cyclos tat ionary- of-order- 
two  (CS (2 ) )  with  period  T  if  and  only  if: 

i)  the  PDF  for  the  random  variable  x(t)  is  identical  to  that  for  the 
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random  variable  x(t+T)  for  every  t  e  J: 

Pr(t)('5  =  Pxft+T)(,)  ¥  t  e  J, 

ii)  for  every  t,s  e  J  the  joint  PDF  for  the  random  variables  x(t) 
and  x(s)  is  identical  to  that  for  the  random  variables  x(t+T)  and 
x(s+T) : 

px(t)xls)^‘*’^  =  Px (t+T)x (s+T) ^ ^  *  t,s  e  J- 

DEFINITION:  A  random  process  (x(t);  t  e  J}  is  cyclfl&I ationary -in -the 
wide -sense  (WSCS)  with  period  T  if  and  only  if: 

i)  the  mean  function  for  the  random  process  x  is  T-periodic: 

m  (t+T)  =  E{x(t+T) }  =  m  (t)  ¥  t  e  .J, 

A  X 

ii)  for  every  t,s  e  J  the  autocorrelation  function  for  the  random 
process  x  is  jointly  T-periodic  in  t  and  s: 

k  (t+T.s+T)  =  E{x(t+T)x* (s+T) }  =  k  (t,s)  ¥  t,s  e  J. 

AA  XX 

Notice  that  condition  ii)  in  this  last  definition--that  kxx(-,-) 
be  jointly  T-periodic  in  its  two  variables--is  less  restrictive  than  the 
condition  that  kxx ( • » • )  be  individually  T-periodic  in  each  of  its 
variables: 

kxx(t+T,s)  =  kxX(t,s+T)  =  kxx(t’s)  V  t,s  e  J’ 

which  if  substituted  for  ii)  would  define  the  class  of  processes  which 
are  periodic -in -the -wide -sense  (WSP) . 

Notice  also  that  it  is  possible--in  fact,  occurs  frequently  m 
practice- -for  WSCS  processes  to  have  constant  mean  and  variance 
(kxx(t’t)'mx(tJ)  ^ke  WS^  Processes«  an<*  yet  have  autocorrelation 
functions  which  display  periodic  fluctuations. 
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Now,  all  processes  which  are  stationary -of -order-two  are  cyclo¬ 
stationary  -of  -order  -two  with  any  choice  of  period  T,  and  all  wide- 
sense  stationary  processes  are  wide-sense  cyclostationary  with  any  choice 
of  period  T,  and  all  wide-sense  periodic  processes  are  wide-sense 
cyclcstationary  and  are  not2  wide-sense  stationary.  In  summary,  these 
various  classes  of  processes  are  related  as  shown  in  the  Venn  diagram 
of  Figure  (1-1). 

We  will  be  primarily  concerned  with  WSCS  processes  in  this  thesis 
and,  from  this  point  on,  will  refer  to  these  simply  as  CS. 

Most  of  the  cyclostationary  processes  studied  in  this  thesis  are 
derived  from  stationary  processes  which  have  been  subjected  to  some  form 
of  repetitive  operation  such  as  sampling,  scanning,  multiplexing,  or 
time-scale  modulation.  There  is,  in  fact,  a  large  variety  of  mechanisms 
by  which  periodicity  is  imposed  on  a  process  which  mignt  otherwise  be 
stationary.  Following  are  several  representative  examples: 

i)  Sampling  and  synchronous  data  signals.  A  particularly  interesting 
example  of  a  cyclostationary  process  is  the  signal  x  which  results  from 
synchronously  sampling  a  WSS  process  y  and  reconstructing  the  waveform 
with  a  prescribed  interpolating  pulse.  Realizations  of  this  process 
take  the  form: 

00 

x (t )  «  l  y(nT)q(t-nT)  (1-1) 

n=-ot. 

where  (y(nT))  are  the  sample  values  of  the  realization  y(*)>  and  q ( • ) 
is  the  interpolating  pulse  For  example,  the  familiar  sample-and-hold 


2This  is  so  except  for  the  single  degenerate  case  where  k  (t,s)  =  constant 
for  all  t,s  e  J.  X 
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operation  is  characterized  by  a  rectangular  q(- )  of  unit  height  and 
width  T.  A  typical  realization  of  a  sampled -and- held  process  is  shown 
in  Fig.  (1-2).  The  mean  and  autocorrelation  for  the  process  are  given 
by  the  expressions  [2] : 

m  (t)  =  m l  q(t-nT) 
yn 

k  (t,s)  =  l  k  (nT-mT)q(t-nT)q(s-mT)  (1-2) 

n,m 

and  cyclostationarity  of  x  is  easily  verified.  Notice  that,  for  the 
sample-and-hold  pulse,  x  hau  constant  mean  and  variance  but  its  auto¬ 
correlation  function  has  strong  periodic  fluctuations  which  are  shown 
graphically  in  Figure  (1-3).  (The  periodic  fluctuations  of  a  CS  process 
are  displayed  along  lines  parallel  to  the(t  =  s)diagonal.) 

Another  version  of  this  process  is  synchronous  pulse-amplitude- 
modulation  (PAM)  where  the  numbers  (y(nT)}  =  {y^}  are  realizations  of  a 
random  data-sequence,  and  the  pulse  shape  q(-)  is  selected  to  minimize  the 
ill  effects  of  intersymbol-interference  [2]  which  result  from  transmission 
over  a  dispersive  communication-channel.  This  model  can  be  made  more 
realistic  by  modifying  the  time-position  of  each  pulse  with  a  random 
jitter  variaole: 

x(t)  =  l  ynq(t-nT+6n). 
n 

The  resultant  process  termed  jittered  PAM  can  still  be  cyclostationary 
with  period  T  [2]  (See  Sec.  7  of  Chapter  II). 

Other  pulse-modulation  techniques  such  as  frequency-shift-keying, 
phase-shift-keying,  pulse-width-modulation,  and  puise-position- 
modulation  also  give  rise  to  synchronous  pulse-trains  which  can  be 
modeled  as  CS  processes  110]  (see  Sec.  4  of  Chapter  II).  In  fact  even 


asynchronous  pulse-trains  can  be  modeled  as  CS  processes  as  shown  in 
Sec.  2  of  Chapter  II. 
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ii)  Scanning  and  time-division-multiplexed  signals.  In  communications, 
one  often  encounters  processes  formed  by  interleaving  finite -length 
records  from  a  multiplicity  of  signal  sources.  As  an  example  consider 
the  process  with  realizations  of  the  form 


x(t)  =  I  xn(t)q(t-nT) 
n=-°° 


(1-3) 


where  q(t-nT)  is  a  "gating  pulse"  (evaluated  at  time  t)  which  "turns  on" 
the  n**1  signal  source  in  the  time-interval  [nT,  (n+l)T],  and  (xn>  are  the 
various  random  signals.  Specifically,  consider  the  time-division- 
multiplex  (TDM)  of  M  different  jointly  WSS  signals  {xn;  x^  =  xn+M  V-  n't 
with  crosscorrelations  k  „(t-s)  =  E{x  (t)x.(s)}  and  means  m  =  E{x  (t)}. 
A  typical  realization  of  such  a  TDM  signal  is  shown  in  Fig.  (1-4).  The 
mean  and  autocorrelation  for  the  composite  TDM  process  are  given  by  the 


expressions : 


■x(t)  -  l  mnq(t-nT) 
n 


kxx(t,s)  =  ^  kn£ (t-s Jq (t-nT)q (s-£T) 

n.K, 


(1-4) 


and  the  cyclostationarity  (period  MT)  is  easily  verified.  Notice  that 
regardless  of  the  differences  among  the  various  auto-  and  cross¬ 
correlation  functions  for  the  M  signals,  if  the  means  and  variances  are 
equal,  the  mean  and  variance  of  the  composite  cyclostationary  process 
are  both  constant  (for  a  rectangular  gate  function).  Yet  its  auto¬ 
correlation  function  can  have  pronounced  periodic  variations  even  when 
all  M  of  the  signals  are  statistically  identical  (in  which  case  the  period 
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of  cyclostationarity  is  T).  Other  CS  models  for  TDM  signals  are  derived 
in  Sec.  2  of  Chapter  II. 

As  another  special  case  of  those  processes  which  take  the  form  of 
Equation  (1-3),  consider  the  video  signal  which  results  from  conventional 
line-scanning  of  a  rectangular  visual -field  [11].  Here  the  {xn>  represent 
consecutive  one- line  segments  of  the  video  signal  and  M  is  the  number  of 
lines  per  frame.  Using  the  Markov  model  derived  by  Franks  [11]  for  the 
line  segments  (see  Figure  (1-5))  we  obtain 

k  (t-s)  =  pln-£le'2irfo|t_s|mod  T,  (t-nT) ,  (s-tT)  e  [0,T] . 
nfc 

Thus,  the  video  autocorrelation  function  becomes  (from  Equation  (1-4)): 

k  (t,s)  =  e"27Tf°lt‘Siri0d  T  l  pln_£l  q(t-nT)q(s-£T)  (1-5) 

n,£ 

which,  interestingly,  is  the  autocorrelation  function  of  a  sampled  and 
held  process  (Equation  (1-2))  modified  by  a  factor--call  it  kj(t-s)-- 
which  is  T-periodic  in  t-s..  This  periodic  factor  is  shown  graphically 
in  Fig.  (1-6),  where  the  parameter  f0  has  been  chosen  much  greater  than 
1/T  as  it.  is  for  typical  viewing  material.  See  Secs.  3,6  of  Chapter  II 
for  a  more  detailed  model  of  the  video  process  which  takes  into  account 
frame-to-frame  correlation  as  well  as  line-to-line  correlation. 

Many  other  CS  processes  such  as  amplitude-modulated  signals,  analog  and 
digital  phase  and  frequency  modulated  signals,  frequency-division-multiplexed 
signals,  etc.  are  introduced  and  discussed  in  Chapter  II. 
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3.  Historical  Notes 

Interest  in  cyclostationary  random  processes  seems  to  have 
originated  in  the  investigations  of  a  group  of  communications  engineers 
and  scientists  im'olved  in  the  development  of  pulse-code-modulation 
(PCM)  at  Bell  Telephone  Laboratories3  in  the  late  1940's  [4].  However, 
the  cyclostationarity  of  these  PCM  processes  did  not  specifically  begin 
to  receive  attention  until  the  late  1950's.  In  1958,  Bennett  [1] 
developed  a  statistical  model  for  the  synchronous  pulse-train  of  PCM  and 
demonstrated  that  the  process  is  nonstationary  and  exhibits  periodically 
varying  mean  and  autocovariance  functions.  He  coined  the  term 
"cyclostationary"  to  denote  the  class  of  nonstationr ry  processes  which 
exhibit  these  cyclic  characteristics,  and  he  introduced  the  Fourier  series 
representation  for  the  mean  and  autocovariance  functions.  In  addition, 

Bennett  discussed  the  phase-randomization  method  of  obtaining  a  stationary 
model  for  a  cyclostationary  process,  and  he  pointed  out  that  the  power 
spectral  density  obtained  from  this  model  is  identical  to  that  obtained 
by  Macfarlane  [12]  in  1949  using  Rice's  general  definition  of  the  average 
power  spectral  density  of  a  nonstationary  process  [13]. 

Four  years  prior  to  Bennett's  publication,  Deutsch  [8]  presented  a 
paper  on  the  envelope  detection  of  cyclostationary  processes --which  he 
referred  to  as  "periodically-stationary"--composed  of  one  or  more 
periodic  signals  each  amplitude  modulated  with  a  stationary  Gaussian 
process.  Although  this  article  preceded  Bennett's,  it  did  not  deal 
primarily  with  the  periodic  structure  of  the  cyclostationary  processes. 

6  The  basic  ideas  of--and  original  patents  on--PCM  date  back  to  the  early 
1930's.  See  Techniques  of  Pulse  Code  Modulation  for  a  historical  review  [39]. 
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At  about  the  same  time  (late  1950's),  interest  in  cycloStationary 
processes  was  initiated  in  Russia  with  a  series  of  theoretical  works  on 
radiophysics.  Gudzenko,  motivated  by  these  works,  published  a  paper  in 
1959  on  the  general  structure  of  cyclostationary  processes  which  he 
referred  to  as  "periodically  nonstationary"  [14].  He  briefly  examined 
the  periodic  correlation  function  and  corresponding  "time-varying 
spectrum"  for  the  general  (arbitrary)  cyclostationary  process,  and  gave 
sufficient  conditions  for  the  consistent  estimation  of  the  coefficient 
functions  in  the  Fourier  series  expansion  of  the  autocorrelation  function. 

He  also  mentioned  the  interpretation  of  the  zero111  erder  Fourier 
coefficient  as  the  autocorrelation  function  of  a  stationary  process,  and 
briefly  discussed  linear  filtering  of  cyclostationary  processes. 

Following  Gudzenko's  paper  were  two  works  published  by  Gladyshev  who 
was  associated  with  the  Institute  of  Atmospheric  Physics  in  the  Academy 
of  Sciences  of  the  USSR  [15,16].  The  first  paper  (1961)  was  an 
investigation  of  the  mathematical  structure  of  cyclostationary  sequences, 
and  the  second  (1963)  was  a  more  brief  investigation  of  the  properties 
of  continuous-time  cyclostationary  and  "almost  cyclostationary"  processes. 
Specifically,  he  considered  the  non-negative  definiteness  of  the  Fourier 
series  coefficient  functions,  and  the  harmonizability  of  these  processes 
which  he  referred  to  as  "periodically  and  almost  periodically  correlated". 

Following  these  pioneering  works,  there  have  been  a  number  of 
publications  dealing  with  cyclostationary  processes  as  related  to 
meteorology,  signals  for  communication,  and  noise  in  electrical  devices 
and  circuits.  Jordan  (1961)  derived  the  least-squares  series  representation, 
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on  a  finite  interval , of  a  random  signal  in  the  presence  of  noise,  and 
briefly  discussed  its  application  to  the  problem  of  least-squares 
estimation  of  cyclostationary  signals  in  noise  [25].  Monin  (1963) 
suggested  using  cyclostationary  processes  as  models  for  meteorological 
processes  [17].  Willis  (1964),  motivated  by  a  study  of  cosmic  ray 
extensive  air  showers,  investigated  a  non-homogeneous  Poisson  process 
witli  periodic  rate  parameter,  and  indicated  how  the  amplitude  and 
frequency  of  the  periodic  rate -of -occurrence  of  events  can  be  estimated 
from  measured  time  intervals  between  events  [18].  Parzen  (1962)  suggested 
that  a  non-homogeneous  Poisson  process  with  periodic  rate  parameter 
would  serve  as  a  suitable  model  of  electron  emissions  from  the  cathode 
of  a  temperature -limited  diode  with  alternating  filament -current  [22]. 
Markelov  (1966)  studied  some  statistical  properties  of  the  cyclostationary 
output  of  a  parametric  amplifier  driven  by  a  stationary  process.  He 
derived  the  average  number  of  axi s -crossings  and  relative  staying  time  in 
an  interval  [21].  Anderson  and  Salz  (1965)  derived  the  power  spectral 
density  for  the  stationarized  version  of  a  cyclostationary  digital  FM 
signal  [23],  and  Lundquist  (1969)  similarly  derived  the  power  spectral 
density  for  digital  PM  [24], 

Several  additional  references  to  work  related  to  cyclostationary 
processes  can  be  found  in  the  references  in  Brelsford's  doctoral 
dissertation  "Probability  Predictions  and  Time  Series  with  Periodic 
Structure",  (1967)  [19].  Chapter  V  of  his  dissertation  is  devoted  to 
a  brief  review  of  representations  for  cyclostationary  processes  (mostly 
sequences)  and  the  presentation  of  autoregressive  techniques  for  the 
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linear  prediction  of  cyclostationary  sequences.  In  particular, 

Brelsford  (and  Jones  [20])  demonstrated- -using  meteorological  data-- 
that  autoregressive  techniques  which  employ  periodically  varying 
coefficients  can  yield  better  predictions  of  cyclostationary  sequences 
than  prior  techniques  which  used  only  constant  coefficients. 

A  few  other  references  to  work  related  to  cyclostationary  processes 
can  be  found  in  the  bibliography  and  historical  notes  in  Hurd's  doctoral 
dissertation  "An  Investigation  of  Periodically  Correlated  Stochastic 
Processes"  (196*)  [5],  In  his  dissertation,  Hurd  investigates  (with 
mathematical  rigor)  the  structure  and  experimental  analysis  of  cyclo¬ 
stationary  processes  which  he  refers  to  as  "periodically  correlated". 

His  work  centers  around  the  Fourier  series  representation  for  the 
correlation  functions  for  cyclostationary  processes,  and  is  closely  related 
to  the  pioneering  works  of  Gudzenko  (1959)  and  Gladyshev  (1963).  Hurd's 
dissertation  is  divided  L.to  two  main  areas.  The  first  contains  an 
examination  of  the  general  mathematical  structure  of  cyclostationary 
processes  and  their  correlation  functions,  and  contains  a  brief 
discussion  of  the  stationarizing  effect  of  a  random  phase  and  the  effects 
of  linear  filtering,  and  includes  a  few  models  for  the  generation  of 
cyclostationary  processes  from  stationary  processes.  The  second  area 
contains  treatments  of  several  topics  in  experimental  analysis.  Specifically, 
cyclostationary  processes  are  shown  to  have  "ergodic"  properties  in  the 
sense  that  the  autocorrelation  function  and  its  Fourier  series 
coefficient -functions  can- -under  certain  conditions--  be  estimated  from 
one  sample  function  of  a  cyclostationary  process.  Also,  some  methods  for 
testing  observed  waveforms  for  cyclostationarity  are  presented  along  with 
some  experimental  results. 
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The  textbooks  of  Stratonovich  (1963)  |7],  Papoulis  (1965)  [3], 

Bendat  and  Piersol  (1966)  [9],  and  Franks  (1969)  [2],  define  the  class 

i 

of  cyclostationary  processes,  give  various  examples,  and  (except  [9]) 

5  1 

discuss  the  stationarizing  effect  of  phase-raridomization.  The  most 

\  * 

extensive  treatment  is  that  of  Franks.  In  a.  chapter  devoted  most'ly  to 

the  development  of  models  for  cyclostationary  processes,  he  points  out 
>  ■  •  > 
that  most  scanning' operations,  such  ?is  radar  antenna  circular-scanning 

and  video  line-scanning,  can  produce  cyclostationary  signals,  and  in 

fact  that  many  of  the  commonly  used  repetitive  signal  processing  ^ 

i 

operations  including  sampling,  scanning,  and  multiplexing  can  transform 
stationary  processes  into  cyclostationary  processes.  He  develops,  in 

detail,  a  cyclostationary  model  for  synchronous  pulse-amplitude- 

!  ' 

modulation  (PAM)  and  shows  that  "jittered"  PAM  (non-synchronous)  can 

I  I  ' 

also  be  cyclostationary.  He  also  introduces  cyclostationary  models  for 

1  |  i  , 

the  time-division  multiplex  of  two  stationary1 signals,  and  for  synchronous 

* 

pulse- width-modulation. 

'  Finally,  we  mention  the  very  recent  work  of  Ogura  (1971)  in  which 

■  i 

he  briefly  discusses  series  and  integral  representations  for  cyclostationary 
processes,  and  presents  in  some  detail  an  interesting  harmonic  series1 

i 

•  J  I  i  5 

representation  for  cyclostationary  processes  and  their  autocorrelation 
functions.  He  also  briefly  discusses  the  measurement  of  the  coefficients 

,  ,  ,  I  ! 

in  his  harmonic  series  representation  for  "ergodic"  cyclostationary 

1 

processes,  and  concludes  with  three  examples  pf  cyclostationary 
processes:  AM,  PAM,  and  PPM. 
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In  summary,  the  major  works  which  deal  directly  with  the  general 
class  of  cyclostationary  processes  are,  in  chronological  order,  those 
of:  Bennett  (1958),  Gudzenko  (1959),  Gladyshev  (1961,  1963), 

Brelsford  (1967),  Franks  (1969),  Hurd  (1969),  and  Ogura  (1971). 

4.  Preliminary  Comments  on  Mathematical  Rigor 

Although  most  engineers  are  familiar  with  and  use  the  Riemann 
integral  [26]  rather  than  the  Lebesgue  integral  [27],  it  is  the  latter 
which  is  indispensable  in  analyses  where  questions  of  the  existence  of 
integral  and  series  representations  of  functions  (random  and  deterministic) 
arise,  and  where  the  interchange  of  the  or.der  of  execution  of  integration, 
infinite  summation,  and  statistical  expectation  must  be  justified. 

(See,  for  example,  Wiener  [28]  pp.  4-5,  and  Epstein  [29]  p.  29).  Such 
questions  of  existence  and  needs  for  justification  arise  throughout  every 
remaining  chapter  of  this  dissertation,  and  can  only  be  rigorously 
dealt  with  in  terms  of  Lebesgue  integration  theory  and  associated 
measure-theoretic  notions.  For  example,  the  work  of  Hurd  [5],  which  is 
closely  related  to  this  thesis,  is  thoroughly  based  in  measure  theory. 

Despite  these  comments,  it  is  not  my  intent  to  clutter  the 
presentation  of  the  methods,  techniques,  and  results  of  this  thesis  with 
rigorous  justifications  of  the  numerous  manipulations,  and  with  precise 
statements  of  the  conditions  under  which  the  various  (known) 
representations  are  valid.  Such  an  undertaking  would  surely  transform 
this  engineering  thesis  into  a  mathematical  work  of  limited  appeal  to 
most  engineers. 
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Rather,  I  will  assume  that  all  random  processes  of  interest  are 
harmonizable  in  the  sense  that  the  double  Fourier  transform  of  their 
autocorrelation  functions  exist  (with  the  possible  inclusion  of  i&pulse 
functions);  that  all  periodic  functions  (including  periodic  trains  of 
impulses)  possess  Fourier  series  representations;  and  that  wherever 
necessary,  the  order  of  execution  of  integration,  infinite  summation, 
and  statistical  expectation  can  be  interchanged.  Clearly  then,  all  proofs 
based  on  these  assumptions  will  be  formal  proofs. 

I  refer  the  mathematically  inclined  reader,  interested  in  the 
validity  of  these  assumptions,  to  the  following  references  chosen  on 
the  basis  of  their  potential  appeal  to  engineers: 

(1)  Lebesgue  integration  theory:  Riesz  and  Sz-Nagy  [31],  and  Epstein  [29], 

(2)  Properties  of  the  Lebesgue  integral,  and  development  of  series  and 
integral  harmonic  representations  for  deterministic  functions:  Wiener  [28], 
and  Titchmarsh  [32,33], 

(3)  Theory  of  linear  integral  equations:  Riesz  and  Sz-Nagy  [31], 

(4)  Theory  of  Distributions  (impulse  functions  and  their  relatives): 
Zemanian  [30] , 

(5)  Development  of  series  and  integral  representations  for  random  functions 
(introductory  treatments  for  engineers):  Pugachev  [34],  and  Papoulis  [3], 

(6)  Linear  functional  analysis:  Epstein  [29],  and  Riesz  and  Sz-Nagy  [31]. 

Finally,  I  mention  that  the  use  of  the  Fourier  transforms  and 
Fourier  series  (except  in  Chapter  III,  Section  2h  where  a  generalized 
Fourier  transform  for  random  processes  is  defined)  and  the  use  and 
manipulation  of  infinite  integrals,  infinite  series,  and  impulse  functions 
in  this  dissertation  are  not  uncommon  in  the  engineering  literature,  and 
are  often  easy  to  justify  heurist ically. 


Figure(l-l)  Venn  diagram  of  various  classes  of  random  processes 
NS  »  generally  nonstationary,  WSCS  »  wide-sense  eye lost at ion ary 
WSS  «  wide-sense  stationary,  CS(2)  =  cyclostationary-of-order- 
two,  S(2)  *  stationary-of-ordcr-two,  WSP  «  wide-sense  periodic. 
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CHAPTER  II 

TRANSFORMATION ,  GENERATION,  AND  MODELING 
OF  CYCLOSTATIONARY  PROCESSES 

1.  Introduction 

This  chapter  provides  a  foundation  upon  which  man/  of  the  develop¬ 
ment*  in  this  thesis  are  based,  and  upon  which  subsequent  studies 
dealing  with  cyclostationary  processes  can  be  based.  We  consider  here 
the  generation  of  cyclostationary  processes  and  transformations  on  them. 
We  develop  formulas  which  describe,  in  terms  of  input-output  relations, 
various  classes  of  transformations  such  as  linear  time- invariant, 
periodically  varying,  and  time-scale  transformations;  and  random  linear 
stationary  and  cyclostationary  transformations.  Also  considered  are 
nonlinear  random  and  deterministic  transformations  encountered  in  various 
modulating  schemes  and  common  models  of  distortion.  We  derive  models  for 
a  multitude  of  cyclostationary  processes  which  are  generated  from 
stationary  processes  that  have  been  subjected  to  transformations  from 
the  above  classes.  We  also  present  a  number  of  theorems  on  transforma¬ 
tions  mostly  related  to  the  generation  and  preservation  of 
cyclostationarity. 

Most  of  the  developments  in  this  chapter  relate  to  the  system 
diagram  shown  in  Figure  2-1  where  y  is  a  random  process  referred  to  as 
the  input  to  the  system  G,  and  x  is  a  random  process  termed  the  output, 
x  is  also  referred  to  as  the  image  of  y  under  the  transformation  G,  and 
y  is  termed  the  inverse-image  of  x. 
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In  Section  2,  we  consider  Transformations  G  which  are  linear. 

After  deriving  general  input-output  relations,  we  individually  discuss, 
in  some  detail,  the  three  subclasses  of  linear  transformations  referred 
to  as  time-invariant  filters  (2a),  periodically  time-varying  systems  (2b), 
and  time-scale  transformations  (2c).  Zadeh's  "system  function"  is 
introduced  as  a  means  for  characterizing  linear  systems,  and  is  used 
throughout  most  of  this  section.  A  number  of  theorems  on  the  properties 
of  these  three  classes  of  transformations  are  proven  and  illustrated 
with  examples  which  also  serve  to  introduce  seven  types  or  classes  of 
cyclostationary  processes  including  time-division-multiplexed  signals, 
frequency-division-multiplexed  signals,  and  a-synchronous  pulse  trains 
including  facsimile,  and  telegraph  signals. 

In  Section  3,  we  briefly  discuss  multidimensional  linear 
transformations  as  a  means  for  characterizing  scanning  operations. 

Examples  are  given  to  illustrate  two  common  types  of  scanning,  and  to 
introduce  two  additional  classes  of  cyclostationary  processes  including 
video  signals. 

In  the  fourth  section,  we  introduce  a  generalization  of  Wiener's 
"Volterra  series  representation"  for  characterizing  input-output  relations 
for  periodic  nonlinear  systems.  Two  subclasses  of  periodic  Volterra 
systems  are  considered,  and  theorems  for  each,  on  the  generation  of 
cyclostationary  processes,  are  proven  and  illustrated  with  examples 
which  also  serve  to  introduce  several  additional  classes  of  cyclo¬ 
stationary  processes,  including  synchronous  pulse-trains  such  as 
frequency-shift-keyed  signals. 
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Section  5,  on  random  linear  transformations,  parallels  Section  2 
on  deterministic  linear  systems.  After  deriving  general  input-output 
relations  in  terms  of  a  generalization  of  Zadeh's  "system  correlation 
function",  we  individually  discuss,  in  some  detail,  the  three  subclasses 
referred  to  as  wide-sense-stationary  systems  (5a),  cyclostationary  systems 
(5b),  and  random  time-scale  transformations  (5c).  A  number  of  theorems 
on  the  preservation  and  generation  of  cyclostationary  processes  are 
proven  and  illustrated  with  examples  which  also  serve  to  introduce 
another  eight  classes  or  types  of  cyclostationary  processes,  including 
analog  and  digital  frequency-modulated  and  phase-modulated  signals. 

Section  6,  on  random  multidimensional  linear  transformations, 
parallels  Section  3.  The  major  application  of  the  theory  in  this  section 
is  to  random  scanning. 

The  final  section  (7),  on  random  nonlinear  transformations,  is  an 
extension  of  Section  4  on  deterministic  nonlinear  systems.  The  major 
issue  in  this  section  is  "jitter".  This  random  disturbance  is 
characterized  in  general  terms,  and  shown  to  preserve  cyclostationarity 
under  a  fairly  liberal  set  of  conditions. 

2.  Linear  Transformations 

Due  to  the  length  of  this  section,  we  begin  by  giving  a  brief 
outline  of  the  theorems  proved  and  the  cyclostationary  process  models 
defined: 

a)  Time-invariant  filters 

Theorem(2-l) :  Preservation  of  cyclostationarity  by  time-invariant  filters. 
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Theorem (2-2) :  Sampling  theorem  for  nonstationary  processes. 

Theorem  (2-3) :  Reduction  of  cyclostationary  processes  to  stationary 
processes  via  bandlimiting  filters. 

b)  Periodically  time-varying  systems 

Theorem(2-4) :  Generation  of  cyclostationary  processes  from  stationary 
processes  via  periodic  systems. 

Model (1):  Amplitude-modulated  signals 
Model  (2):  Pulse-amplitude-modulated  signals 
Model (3):  Time-division-multiplexed  signals 
Model  (4) :  Frequency-division-multiplexed  signals 

c)  Time-scale  transformations 

Theorem (2-5 ) :  Generation  of  cyclostationary  processes  from  stationary 
processes  via  periodic  time-scale  transformations. 

Model  (5):  Time-division-multiplexed  signals 
Model  (6):  Doppler  shifted  processes 
Model  (7):  Random  facsimile  signal 

Model  (8) :  Asynchronous  pulse-amplitude  modulated  signals 

We  now  turn  to  the  development  of  general  input-output  relations  which 
will  be  employed  throughout  the  remainder  of  this  thesis.1* 


4Throughout  most  of  this  dissertation,  we  will  use  single  letters, 
say  x,  to  denote  random  processes,  and  we  will  use  the  form  x(t)  to 
denote  a  realization  of  x.  Similarly,  we  will  use  the  notation  y(t), 
z(t,s)  to  denote  deterministic  functions  of  one  and  two  variables. 

However,  on  occassion,  the  notation  y(’),  c(*,*)  will  be  used  for  the 
purpose  of  clarification.  The  index  sets  for  random  processes,  and  domains 
for  deterministic  functions  will,  unless  otherwise  stated,  be  (-»,“>) 
or  cartesian  products  thereof. 


All  linear  transformations  of  interest  in  this  thesis  are  linear 
integral  transformations  and  can  be  characterized  by  an  impulse-response 
function  (kernel)  g(t,t),  which  may  possess  impulse  functions  and  their 
derivatives.  Thus  any  realization  x(t)  of  the  output  of  G  can  be  related 
to  its  inverse-image  (input)  y(t)  as  follows:5 


x(t)  ■  /  g(t,T)y(T)dx. 


(2-1) 


*  /  g(t,t)m  (t)dT, 

.00  ' 


The  mean  function  for  x  is  related  to  the  mean  function  for  y  as  follows: 

00 

mx(t)  -  E{x(t) }  =  E{  /  g(t,T)y(T)dt) 

-00 

00 

=  /  g(t,T)E{y(T)}dT 

(2-2) 

and  the  autocorrelation  function  for  x  is  related  to  that  for  y  as  follows 

cr» 

kxx(t,s)  "  E{x(t)x*(s)}  =  E<  //  g(t»T)g*(s,Y)y(x)y*(Tf)dTdy} 

—  00 

00 

=  //  g(t,i)g*(s,y)E{y(T)y*(Y)}dTdY 

.00 

00 

=  //  g(t,T)g*(s,y)k  (T,Y)didy.  (2-3) 

.oo  y  y 

Fourier  transform  of  the  autocorrelation  function 
(sometimes  referred  to  as  tne  cointensity  spectrum)  is  a  quantity  which 
finds  much  application  in  the  theories  of  representation  and  estimation 
of  random  processes.  We  denote  the  double  Fourier  transform  of  kxx(t,s) 

as  *xx(f'v): 


5This  characterization  is  generalized  in  Section  3  of  this  chapter  to 
multidimensional  transformations . 
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K  :(f,v)  $  //  k  v(t,s)e'j27r(ft'v^dtds. 


(2-4) 


We  can  express  the  double  Fourier  transform  of  a  function  of  two  variables 
in  terms  of  a  repeated  single  Courier  transform  as  follows:  , 


let  &(t,T)  *  « /  g(t,T)e‘j2*ftdt;  " 

-00 

1 

OO  . 

then  •  G(f ,  v)  =  /  2f(f, +)e3,dT. 


With ‘the  aid  of  this  definition  of  &  we  can  easily  relate  Kxx(^»v) 
directly  to  K  (f,v)  by  combining  Eqs.  (2-3),  (2-4): 

i 

00  ' 

K  Cf,v)  x  ////  k  (T,Y)g(t,T)g*(s,y)e  ^2ir^t  vS^dTdydtds 
-oo  yy  ; 

00 

=  //  k  (T,Y)G(f,T)&*(v,Y)dTdy  .  1  , 

-00  ,  *  *  ! 

!  I 

Now,  using  Parseva.l's  relation  [2]  we  have 


K  (f,v).  =  //  K  (w,cj){  /  &*(f,T)e"j2™TdT}*- 

-  »  /  I  _  00 


(  /  ^(v,Y)e^'27TYadY}dtodo 


=  //  K  («o,0)G(f ,cu)G* (v,o)daida . 
.»  . yy  i 


(2-5) 


Notice  that  Eq.  (2-5)  is  the  "frequency-time  dual"  of  Eq.  (2-3). 

,  i  t 

There  are  two  interesting  subclasses  of  linear  transformations  that 

i 

are  dealt  with  throughout  this  thesis.  We  describe  these  in  the  next 

I 

two  subsections.  . 

a)  Time  inyariant  filters.  The  subclass,  time-invariant  linear 
transformations --frequently  referred  to  as  filters--is  characterized  by 
the  defining  relation  (with  some  abuse  of  notation)  g(t,r)  =  g(t-i) 
for  all.  t  and  t..  That  is,  the  impulse -response  function  foxj  a  linear 


time-invariant  filter  depends  only  on  the  difference  of  its  arguments 
and  is,  in  fact,  a  function  of  a  single  variable.  The  double  Fourier 
transform  of  g(t--r)  exhibits  the  following  singular  behavior: 
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G(f,v)  =  G(f)6(f-v),  (2-6) 

where  G(f)  is  the  single  Fourier  transform  of  g(t)  and  is  referred  to 
as  the  transfer  function  for  the  filter,  and  6  is  the  Dirac  delta 
function  (impulse  function). 

Using  Eq.  (2-6),  the  input-output  relation  of  Eq.  (2-5)  reduces  to 

Kxx(f,v)  =  Kyy(f,v)G(f)G*(v).  (2-7) 

Now,  if  y  is  WSS  then  its  autocorrelation  function  depends  only  on  the 
difference  of  its  arguments  and,  as  a  result, 

Kyy(f,v)  =  Kyy(f)6(f-v)  (2-8) 

where  Kyy(f)  *s  the  single  Fourier  transform  of  the  autocorrelation 
function  kyy(t)  and  is  referred  to  as  the  power  spectral  density  (PSD) 
for  y  (and  sometimes  as  the  autointensity  spectrum).  Using  Eq.  (2-8)  in 
Eq.  (2-7)  yields 

K  (f,v)  =  K  (f)6(f-v)G(f)G*(v) 

aa  yy 

=  Kyy(f) | G (f ) 1 26(f-v)  . 

Also,  if  y  is  WSS  then  its  mean  function  is  a  constant  and  (from  Eq.  (2-2) 
with  g (t , t)  =  g(t-x))  so  too  is  the  mean  function  for  x.  Thus,  x  is  WSS 
and,  from  the  last  equation,  has  PSD 
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KXX(f)  "  lG^|2lCyy(£).  (2-9) 

The  following  theorem  is  an  intuitively  obvious,  but  useful,  result 
on  the  preservation  of  the  property  of  cyclostationarity  by  time- 
invariant  filters: 


THE0REM(2-1):  If  the  input  to  an  arbitrary  time-invariant  filter  is  a 
cyclostationary  process  with  period  T,  then  the  output  is  cyclostationary 
with  period  T. 

Proof:  From  Eq.  (2-3)  we  have 

00 

kxx(t+T*s+T)  =  //g(t+T-r)g*(s+T-y)k  (x,Y)dxdY 
-  00  '  ' 

00 

=  //g(t-x')g*(s-Y')k  (x'^T,’y,+T)dx'dY' 

_  OO  •  ' 

00 

=  // g(t-x’)g*(s-Y')k  (x'.Y'Jdx'dY' 

-00  '  ' 

=  k  (t,s)  V  t,s  , 

XX  » 


and  from  Eq.  (2-2) 

00 

m  (t+T)  =  /  g(t+T-x)m  (x)dx 

a  y 

-oo  ' 

00 

=  /  g(t-x')m  (T’+T)dT' 

-00  ' 

00 

=  /  g(t-T')m  (x')dT' 

-00  ' 

=  m  (t)  V  t  ' 

xv 


Hence,  both  the  mean  and  autocorrelation  for  x  are  T-periodic  so  that 
x  is  T-CS.6 

QED 


6We  will  use  the  short-hand  notation  T-CS  to  denote  cyclostationary  (in 
the  wide  sense)  with  period  T. 
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i)  Band limiting. 

(1)  Sampling  Theorem:  The  well  known  sampling  theorem  for  WSS  processes 
[2,3]  may  be  stated  as  follows: 

If  the  PSD  K^ff)  of  a  WSS  process,  x  is  bandlimited  to  the  interval 
(-1/2T,1/2T)  in  the  sense  that  K  (f)  =  0  for  |f j  >  1/2T,  then  x 
admits  the  mean-square  equivalent  "sample  representation" 

E{(x(t)  -  l  x(nT)  )2>  -  0  V-  t. 

n=-°»  1  * 

We  now  state  and  prove  a  generalization  of  this  theorem  which  removes 
the  restriction  to  WSS  processes: 

THEOREM (2- 2) :  If  the  double  Fourier  transform  Kxx(f,v)  of  the  auto¬ 
correlation  function  for  a  process  x  is  bandlimited  to  (-1/2T,1/2T)  in 
the  sense  that  Kxx(f»v)  =  0  if  |f|  >  1/2T  or  if  |v|  >  1/2T,  then  x 
admits  the  mean-square  equivalent  representation 


00 

E{ (x(t)  -  l  x(nT) 


n=-°° 


sinx(t-nT)/T  _  0 
7r(t-nT)/T  ; 


V-  t. 


Proof : 


,  _  A  ...  r  simr(t-nT)/T  ,2i 

Let  £  -  E{(x(t)  -  2,  x(nT)  * 

n=-oo  1  J/ 

Expanding  the  square  and  interchanging  expectation  and  summation  yields 


e  =  kxx(t,t)  -  2^  kxx(t,nT)*(t-nT) 
n 

+  l  kxx(nT,mT)4>(t-nT)<|>(t-mT) 


A  simtt/T 
rrt/T 


where 
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Since  k^Ct^s)  is  the  inverse  double  Fourier  transfora  of  K^Cf ,v) ,  then 

k^Ct.s)  =  //  Kxx(f,v)ej2ir(ft"vs)dfdv 

=  if27  K  (f,v)e*2ir(ft'vs)dfdv, 

-1/2T 

and  the  last  equality  is  a  result  of  the  bandliaiting  hypothesis. 

Substituting  this  last  equation  into  the  last  expression  for  e 
we  obtain 

£  =  If'"  «  (f.v)[ei2"(f-v)t  -  2  l  ej2'(ft-VnT)*(t-nT) 

-1/2T  n 

+  l  ej2ir(fnT'VmT)(Kt-nT)<Kt-mT)]dfdv. 
n,m 


But  using  the  Poisson  sum  formula  [2]  we  have 


l  e~j2irvnT<j,(t_nT)  =  ~l  *U/T-v)e~’12lt(v")l/T)t 
n  i 


where 


jf |  <  1/2T 

If  I  >  1/2T  , 


so  that 

l  e‘j2TrvnT<Ht-nT)  =  e"j2lTVt  ,  |v|  <  1/2T. 

n 

Now,  substituting  this  equation  into  the  last  expression  for  e  yields 


1/2T 


£  —  f]  K  (f,v)[ej2"tf-'')t-2ej2"Cf-v)t.ei2'(f-v)t]dfdv 

1  /  'VTl  XX 


•1/2T 
=  0 


Y  t. 


QfcD 

Notice,  from  liq .  (2-7),  that  the  bandlimiting  constraint  on  Kxx(f,v) 
in  the  above  theorem  is  satisfied  for  a  process  x  which  is  the  output  of 
any  time -invariant  filter  whose  transfer  function  is  bandlimited  to 
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(-1/2T,  1/2T)  regardless  of  the  input  process  y.  Notice  also  that  if  x 
is  WSS  then  the  band limiting  constraint  in  the  above  theorem  becomes 


K^Cf.v)  =  K^CfWf-v)  =  0 

if  If  I  >  1/2T  or  if  |v|  >  1/2T  and  is  satisfied  if  and  only  if  ^^(f)  =  0 
for  ]f|  >  1/2T  which  is  precisely  the  bandlimiting  constraint  in  the 
sampling  theorem  for  WSS  processes.  Hence,  the  sampling  theorem  for 
WSS  processes  is  simply  a  special  case  of  the  general  sampling  theorem 
presented  here. 

(2)  Reduction  of  cyclostationary  processes.  Any  CS  process  can  be 
reduced  to  a  WSS  process  via  bandlimiting.  This  fact  is  stated  here 
as  a  theorem: 

THEOREM(2-3) :  If  x  is  the  output  of  a  filter  with  transfer  function 
bandlimited  to  (-1/2T.1/2T)  and  the  input  y  is  T-CS,  then  x  is  WSS. 

Proof:  From  Eq.  (2-7)  Kxx(f,v)  is  bandlimited  to  (-1/2T.1/2T) .  Thus, 
from  the  sampling  theorem  for  deterministic  functions  [2],  we  have 


k  ft  s)  =  I  k  fnT  mTI  rsiniI^-nT)/Ti  rSim^s-mT)/!’ 

xx1  ’  J  L  xxl  ’  ,jl  ir(t-nT)/T  J 1  Tr(s-mT)/T  J' 
n,m  ■ 


But,  from  Theorem  (2-1),  x  is  T-CS  so  that 


kxx(nT,mT)  =  kXJ{[(n-m)T,0] 


V-  n,m. 


and 


k  (t,s) 
xx 


V  v  ,'nT  ni  rSinrr((t-mT)-pT)  rsimr(s-mT)/T1 
A  xxiP  ’  Jl  it ( (t-mT) -pT)  Jl  7r(s-mT)/T  J 


m,p 


((t-mT)-pT) 

simr(s-mT)/T. 


I  k  (t-mT.O)^1-^^] 
xx  1  ir(s-ir.T)/T  1 


=  k  (t-s,0) 
xx 


V-  t,s , 


where  the  last  two  equalities  are  due  to  the  sampling  theorem  for 
deterministic  functions.  Now,  from  Eq.  (2-2)  it  is  easily  shown  that 
the  Fourier  transform  of  »x(t)  is  band limited  to  (-1/2T.1/2T).  Thus, 
from  the  sampling  theorem  for  deterministic  functions,  we  have 


m  (t)  = 


l  »X(«T)[ 


sin*(t-nT)/T  , 
x(t-nT)/T  J 


=  m  (0) 
x  J 


l 


sinir(t-nT)/T 

x(t-nT)/T 


=  m  (0)  Y  t  , 

x 


where  the  second  equality  is  due  to  the  fact  that  x  is  T-CS  (from 
Theorem  (2-1)),  and  the  last  equality  is  due  to  the  sampling  theorem 
for  deterministic  functions.  Hence,  both  the  mean  and  autocorrelation 
functions  for  x  are  invariant  to  arbitrary  time  shifts  so  that  x  is  WSS. 

QED 

Notice  that  this  last  theorem  does  not,  as  might  appear,  contradict 
Theorem  (2-1)  since  any  WSS  process  is  CS  with  any  choice  of  period  T. 

b)  Periodically  time-varying  systems.  The  subclass,  periodically 
time-varying  linear  transformations  (with  period  T) ,  frequently- -but 
sometimes  inappropriately7- -termed  "periodically  time-varying  filters" 
are  characterized  by  the  defining  relation  g(t,r)  =  g(t+T,T+T)  for  all 
t  and  t.  That  is,  the  impulse  response  (kernel)  for  a  linear  T-periodic 
transformation  is  invariant  to  time-translations  that  are  integer 
multiples  of  the  period  T. 


7In  the  event  that  <he  transformation  is  non-dispersive--it's  impulse- 
response  function  contains  a  factor  of  the  form  6(f(t)-i)  for  some 
function  f (t )  —  it  is  not  a  filter. 
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For  many  analyses  involving  linear  time-varying  systems  (transformations) 
it  is  beneficial  to  separate  the  time-varying  and  tiae-invariant 
characteristics.  The  systea  function  defined  by  Zadeh  [35,36]  and  its 
inverse  Fourier  transfora  do  just  this.  The  systea  function  H  is 
defined  as  follows  : 

CD 

H(t,f)  -  /  g(t,t-x)e'j2  clx,  (2-10) 

.(O 

and  the  inverse  Fourier  transform  w.r.t.  (with  respect  to)  f  of  this 
system  function  is 

h(t,x)  =  g(t.t-T),  (2-11) 

where  x  is  frequently  referred  to  as  the  age  variable  or  memory  variable 
and  t  may  be  referred  to  as  the  fluctuation  variable.  Now,  the  time 
invariant  behavior  is  characterized  by  the  section  function  h(t, •)  where 
t  is  considered  to  be  a  parameter,  and  the  time-varying  behavior  is 
characterized  by  the  section  function  h(*,x)  where  x  is  a  parameter  . 8 

For  example,  if  the  transformation  is  time- invariant,  then 
h(t,x)  =  g(t-(t-x))  =  g(x)  =  h(0,x)  for  every  t  and  there  are  no  time 
fluctuations--!)  depends  solely  on  the  age  variable.  Similarly,  if  the 
transformation  is  T-periodic,  then  h(t+T,x)  =  g(t+T,t+T-x)  =  g(t,t-x)  =  h(t,x) 
for  all  t  and  x,  and  the  periodicity  is  reflected  only  in  the  fluctuation 
variable.  Treating  x  as  a  parameter,  we  can  expand  the  T-periodic 


8A  quantity  closely  related  to  the  system  function  is  the  bifrequency 
function  which  is  simply  the  Fourier  transform  w.r.t.  t  of  Hft,f).  The 
bifrequency  function  finds  much  application  in  analyses  of  time-varying 
systems,  but  will  not  be  used  here. 
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section  function  h(* ,t)  into  a  Fourier  series 

h(t,r)  =  l  hn(T)ej2¥nt/T.  (2-12) 

n=-“ 

Thus,  from  Eq.  (2-11),  the  impulse  response  g  has  the  representation 


g(t,x)  =  l  hn(t-T)ej2inlt/T  (2-13) 

n 

One  interesting  result  of  this  representation  is  the  obvious  realization 
of  a  periodic  system  as  a  parallel  connection  of  time- invariant  filters 
{lO  followed  by  periodic  multipliers  (e^2irnt^T)  as  shown  in  Figure  (2-2). 
Notice  that  if  the  sine-cosine  Fourier  series  were  used  rather  than  the 
exponential  Fourier  series,  then  all  components — sine  and  cosine 
multipliers  and  time- invariant  filters— in  the  realization  would  be  real 
not  complex.  (See  Chapter  III  for  other  realizations  of  periodically 
time-varying  linear  systems  ) 

The  double  Fouiier  transform  of  the  impulse  response  of  a  T-periodic 
system  also  has  an  interesting  representation  which  we  obtain  directly 
from  Eq.  (2-1 6) : 


G(f,v)  =  //  g(t,T)e_:,27,Cft'VT)dtdT 
•  00 

=  //  l  h  (t-,)e-j2"t<f-n/T>ei2,',vdtd, 

-®  n  n 

=  l  f  H (f-n/r)e-j2,T<f-n/T>e22'Tvdtdt 
n-00 

=  l  H  (f-n/T)6(f-v-n/T), 
n 


(2-14) 
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where  is  the  single  Fourier  transform  of  hR.  We  see  that  G(f,v) 
consists  of  impulse  fences  along  lines  parallel  to  the  (f  =  v) -diagonal, 
so  that  the  general  input -output  relation  of  Eq.  (2-5)  reduces  to: 

0»  CO 

K  (f,v)  =  l  ■  l  H  (f-n/  T)H*(v-m/  T) K  (f-n/T , v-m/T)#  (2-15) 
AA  n=-°°  m=-“  n  m  yy 

Notice  that  if  we  consider  the  degenerate  case  of  a  time-invariant 
filter,  then  =  0  for  n  ^  0,  and  HQ(f)  =  G(f)  so  that  Eq.  (2-15) 
reduces  to  Ec|.  (2-9)  as  it  should. 

Now,  if  the  input  y  is  WSS  then,  from  Eqs.  (2-8),  (2-15),  the 
double  Fourier  transform  of  the  output -process  autocorrelation  function 
takes  the  form 

Kxx(f’v)  =  ^  Hn(f_n/  T)H^f-n/T)KvvCf-n/T). 

A  n,m 

6(f-v  -(n-m)/  T) 

=  l  [I  Hn(f-n/  T)H*  (f-n/T)K  (f-n/T)]. 
p  n  y  yy 

6 (f-v-p/T)  .  (2-16) 

Thus,  we  see  that  Kxx(f,v)  also  consists  of  impulse  fences  on  lines 
parallel  to  the  (f  =  v) -diagonal  so  that  k  (t,x)  must  exhibit  the  same 
periodicity  as  g(t,x).  We  state  this  result  as  a  theorem  which  provides 
the  foundation  for  the  generation  of  a  large  class  of  CS  processes: 
THE0REM(2-4) :  If  the  input  y  to  a  T-periodic  system  is  WSS,  then  the 
output  x  is  T-CS. 

Proof:  From  Eq.  (2-3),  we  have 
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kxx(t+T»s+T)  =  //  g(t+T»'r)g*(s+T,Y)kyy(T-Y)dTdy 
00 

=  //  g (t , T-T)g* (s ,Y-T)k  (t-T- (y-T) )dxdY 

_oo  7  / 

00 

=  //  g(t,T')g*(s,Y')k  (T'-Y'idT'dY1 

oo  J  7 

=kxx(t’s)  V't*s* 

and,  from  Eq.  (2-2), 

00 

m  (t+T)  =  /  g(t+T,T)m  (T)dt 

^oo  y 

00 

=  /  g(t,T-T)ra  (T-T)dT 

_  00  ' 

00 

=  /  g(t,T’)m  (T')di' 

_00  * 

=  mx(t)  v-  t, 

where  g  is  the  impulse  response  of  the  T-periodic  system.  Hence,  both 
the  mean  and  autocorrelation  functions  for  x  are  T-periodic  so  that 
x  is  T-CS. 

QED 

We  now  present  four  examples  of  CS  processes  which  can  be  generated 
by  passing  WSS  processes  through  periodic  systems.  These  examples  are 
all  signal  formats  used  in  communication. 

MODEL(l):  Amplitude-modulation  (AM).  Conventional  amplitude-modulation 
is  nothing  more  than  multiplication  of  a  signal  by  a  periodic  waveform  p (t ) — 
usual ly  a  sinusoid.  If  x  is  the  AM  process  obtained  from  the  WSS  signal 
y  as  follows:  x(t)  =  p(t)y(t),  then  wc  see  that  x  is  obtained  from  a 
periodic  linear  transformation  on  y  where  the  kernel  (impulse  response) 
is  g(t,r)  =  p(t)6(t-x).  Thus,  from  Theorem  (2-4),  x  is  CS.  An  interesting 
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special  case  of  AM  is  that  where  y  has  a  zero-mean  value  and  p(t)  is  a 
sinusoid  with  period  T.  Then 

kxx ^t,s)  *  cos(2irt/T)cos(2irs/T)kyy(t-s) 

=  y[cos(2ir(t+s)/T)+cos(2ir(t-s)/T)]k  (t-s), 
l  yy 

so  that  x  is  CS  with  period  T/2  even  though  p(t)  is  T-periodic,  not 
T/2-periodic. 

M0DEL(2):  Pulse-amplitude-modulation  (PAM).  As  discussed  in  Chapter  I, 
a  synchronous  PAM  signal  x  is  simply  a  periodic  train  of  equally  spaced 
(in  time)  pulses  with  random  amplitudes: 

oo 

x(t)  =  l  anqo(t-nT)  . 
n=-c° 

The  random  sequence  {an>  may  be  a  random  data  sequence  with  a  finite 
alphabet  (finite  number  of  admissable  values  for  realizations  of  each  an) 
or  {a^}  may  be  the  sample  valves  (y(nT)}  of  some  continuous-time  process 
y.  In  this  latter  case,  x  can  be  generated  by  passing  y  through  a 
periodic  system  composed  of  a  periodic  impulse  sampler  followed  by  a 
time-invariant  filter  with  impulse  response  qQ(t)  as  shown  in  Figure  (2-3). 
The  impulse  response  for  this  composite  T-periodic  system  is 

g(t,x)  =  q0(t-x)  l  6 (t-nT)  . 
n 

If  y  is  WSS  then,  from  Theorem  (2-4)  x  is  T-CS. 

In  Sec.  7  of  this  chapter  we  discuss  a  generalized  model  of  PAM 
which  includes  a  random  disturbance  termed  jitter.  In  this  generalized 
model,  the  pulse  stream  is  no  longer  synchronous,  but  x  is  still  CS. 

Also,  if  the  pulses  comprising  a  single  PAM  signal  are  sufficiently 
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separated  in  time,  then  several  of  these  signals  can  be  interleaved  in 
time  to  form  a  composite  time-division-multiplex  PAM  signal  which  will 
be  CS  if  the  random  amplitude  sequences  are  jointly  WSS.  Note  that 
processes  which  are  individually  WSS  and  are  statistically  independent 
are  also  jointly  WSS. 

MODEL (3):  Time-division-multiplex  (TDM^).  The  TDM^  process,  as  discussed 
in  Chapter  I,  is  formed  by  interleaving  finite-length  records  from  a 
multiplicity,  say  M,  of  signal  sources.  This  can  be  done  by  passing  each 
component  process  through  a  periodic  gate,  and  summing  the  gated  processes 
as  shown  in  Figure  (2-4a) .  A  typical  periodic  gate -function  is  shown  in 
Figure  (2-4b).  Realizations  of  the  composi-.?  process  take  the  form 
00 

x(t)  =  l  yn(t)qo(t-nTQ) 

n=-~ 

where  Tq  =  T/M  and  y^  =  y^+M  for  all  n.  As  pointed  out  in  Chapter  I, 
x  is  CS  with  period  T  if  the  {yn }  are  jointly  WSS.  This  result  can  be 
explained  in  terms  of  a  vector  formulation  of  Theorem  (2-4)  where  the  WSS 
y  is  replaced  by  an  M-vector  of  jointly  WSS  processes,  and  the  T-periodic 
system  is  replaced  with  a  1  x  M  matrix  of  T-periodic  systems. 
Alternatively,  the  cyclostationarity  of  x  can  be  attributed  to  the  fact 
that  a  sum  of  jointly  T-CS  processes  is  a  T-CS  process. 

Notice  that  the  gating  operation  in  the  multiplexor  discards  portions 
of  each  component  process,  so  that  conceivably  there  could  be  a  loss  of 
infoi'mation  in  this  multiplexing  scheme.  In  the  following  subsection 
on  time-scale  transformations,  we  introduce  a  "lossless"  time-division 


multiplexing  scheme,  TDM^. 
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M0DEL(4):  Frequency-division-multiplex  (FDM).  A  multiplexing  scheme 
which  is  frequently  encountered  in  radio  communication  systems  is  that 
of  frequency-division-multiplex  (FDM).  The  FDM  signal  is  formed  by 
interleaving  in  frequency  a  multiplicity,  say  M,  of  "frequency-translated" 
signals,  and  is  to  some  extent  the  frequency-time  dual  of  TDM. 

Specifically,  each  component  process  yn  is  bandlimited  to  (-1/2T,1/2T) 
and  then  amplitude-modulated  with  a  sinusoid  (translated  in  frequency) 
into  disjoint  frequency  bands,  as  shown  in  Figure  (2-5b),  and  the 
resultant  translates  are  summed  to  form  the  composite  FDM  signal  with 
realizations  of  the  form: 

M 

x(t)  =  £  x  (t)cos(w  t+2imt/T) , 

n=l  L 

where  the  (x^)  are  the  bandlimited  versions  of  the  {y^}.  The  complete 
multiplexor  is  shown  in  Figure  (2-Sa), 

Now,  if  u)q  is  an  integer  multiple  of  2ir/T,  and  the  {x^}  are  jointly 
WSS,  then  x  is  the  sum  of  M  jointly  T-CS  processes  and  is  therefore  T-CS. 
Again,  as  in  the  case  of  TDM,  this  .esult  can  be  interpreted  in  terms  of 
a  vector  version  of  Theorem  (2-4) . 

Notice  that  in  both  TDMj  and  FDM,  the  composite  signal  is  the  sum  of 
CS  AM  processes;  that  is,  the  sum  of  deterministic  periodic  signals 
amplitude-modulated  by  random  WSS  signals. 

c)  Time-scale  transformations.  The  realization  y(t')  of  a  process 
which  has  undergone  a  time-scale  transformation  t'(t)  is  a  function  of  a 
function  y[t'(t)J  and  may  therefore  be  referred  to  as  a  composition.  We 
will  consider  WSS  input  processes  y  which  undergo  time-.cale  transformations 
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i 

consisting  of  a  scaled  identity  function  and  a  periodic  function  p(t), 

1  i 

so  that  the  time  parameter  t  is  replaced  by  t'  =  at  +  p,(t)  to  yield  an 
output  composition-process  x  with  realizations  of  the  form  > 

i 

v  ■ 

X(t)  =  y(at+p(t)).  ’  '  '  (2-17) 


Time-rscale  transformations  of  this  form  can  ,be  characterized  by 

i  * 

their  impulse  response  functions  which1  take  the  form 


g'(t,t)  =  6(f-at-p(t))  '  '  1  (2-18) 


Notice  that 


g (t+T ,'t+T)  ~  g(t,T+(l-a)T) 

t  g(t,t)  if  a  t  1, 


so  that  these  "periodic"  time-scale  transformations  are  not  periodic 
systems  unless  a  =  1.  t  1 

i 

The  following  theorem  serves  as  a  basis  for  the  generation  pf  a 
large  class  of  CS  processes:9  .  , 

THEOREM(2-5) :  If  x  is  a  process  with  realizations  obtained' from  the 

I 

realizations  of  a  WSS  process  y  by  the  time-^scale  transformation: 

| 

x(t)  =  y(at+p(t)),  where  a  is  a  deterministic  constant  and  p(t)  is  a 
deterministic  T-periodic  function,  then  x, is  T-CS  and  has  constant  mean 

I 

and  variance. 

Proof :  _ 

mx(t+T)  =  E{y(a(t+T)+p(t+T))}  '  , 

=  m  ,  a  constant  V"  t 

y 

=  mx(t)  V-  t, 


9This  class  has  also  been  considered  by  Hurd  [5] . 


! 
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and 

kxx(t+T,sVT)  =  E{y  (a  (t+T)  +p  (t+T)  )y  (a  (s+T)+p  (s+T)  )  } 

=  E{y  (a  (t+T)+p  (t)  )y  (a  (s+T)+p  (s)  )  } 

=  (a  (t+T)  +p  (t )  -a  (s+T)  -p  (s ) ) 

=  kyy(at+p(t)-as-p(s)) 

=  k  (t,s)  , 
xx v  * 

so  that  x  is  T-CS  with  constant  mean,  and  variance 


a2  =  k  (t,t)  -  m2(t)  =  k  (0)  -  m2 
xx  x  yy  y 

which  is  also  constant. 

QED 

It  is  worth  emphasizing  that  all  members  of  the  subclass  of  CS 
processes  which  are  generated  from  WSS  processes  via  periodic  time-scale 
transformations  have  constant  mean  and  variance  regardless  of  the  severity 
of  periodic  fluctuations  in  the  transformations. 

We  proceed  with  three  examples: 

M0DEL(5):  Time-division-multiplex  (TDM^).  The  time-division-multiplexing 
scheme  (TDM^)  introduced  in  Chapter  I  and  discussed  earlier  in  this  section 
can  result  in  a  loss  of  information  in  the  sense  that  only  a  fraction  (1/M) 
of  every  T-length  record  of  the  component  processes  is  contained  in  the 
composite  TDM^  process--the  remaining  fraction  ( (M- 1 )/M)  is  discarded  by 
the  gates  in  the  multiplexor  (see  Fig.  (2-4)).  This  loss  of  information 
can  be  circumvented  if  every  T-length  record  is  first  compressed  in  time 
to  a  T/M  -length  record.  The  resultant  periodically  compressed  component 
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processes  can  then  be  multiplexed  together  without  loss  of  information. 

The  periodic  time-scale  compression  can  be  effected  by  applying  a 
time-scale  transformation  to  the  original  component  processes: 
x  (t)  =  yn(t+p(t)).  An  appropriate  function  p(t)  is  the  periodic 
triangular  waveform  shown  in  Figure  (2-6a).  With  this  choice,  the 
transformed  time  variable  t'  =  t  +  p(t)  remains  constant  (time  does  not 
advance)  over  the  first  (M-l)/M  of  each  adjacent  T-length  interval,  and 
then  increases  linearly  with  slope  M  (time  advances  at  a  rate  accelerated 
by  the  factor  M)  over  the  final  1/M  of  each  adjacent  T-length  interval 
(see  Figure  (2-6b)). 

The  resultant  effect  of  this  periodic  time-scale  compression  is 
shown  in  Figure  (2-7)  with  a  typical  realization  of  the  Mth  process 
yM  and  its  compressed  counterpart  xM> 

In  order  that  the  M  processes, to  be  time-division-multiplexed, can 
be  properly  interleaved,  the  component  processes  must  be  compressed  into 
interleaving  time-slots  as  expressed  here: 

xn(t)  =  Yn(t+p(t+(M-n)T/M)). 

Now,  the  compressed  component  processes  can  be  applied  to  the  multiplexor 
of  Figure  (2-4)  to  yield  the  composite  TDM^  signal  with  no  loss  of 
information,  and  with  realizations  of  the  form: 

x(t)  =  l  yn(t+p(t+(M-n)To))qo(t-(n-l)To) 
n 

where  T  =  T/M. 
o 
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If  the  original  component  processes  {y^}  are  jointly  WSS,  then 
each  compressed  component  process  will  be  T-CS  (from  Theorem  (2-5)) 
and,  in  fact,  the  M  components  will  be  jointly  T-CS.  An  extended 
version  of  Theorem  (2-4)  (a  vector  of  jointly  T-CS  inputs  into  a  matrix 
of  T-periodic  systems  yields  a  T-CS  output)  guarantees  that  the  composite 
TDM2  signal  x  is  T-CS. 

Notice  that  any  time-scale  transformation  (including  the  time-scale 
compression  in  the  above  example)  can  be  realized,  ir.  principle  at  least, 
with  a  recording  machine  which  either  records  or  plays  back  (not  both)  at 
a  variable  speed  determined  by  the  time-scale  transformation. 

M0DEL(6):  Doppler  shifted  processes.  A  realization  of  any  random 
waveform— be  it  acoustic,  electromagnetic,  mechanical,  etc. --which  is 
transmitted,  radiated,  or  reflected  from  a  body  whose  spatial  frame  of 
reference  varies  w.r.t.  that  of  an  observer  of  the  random  waveform  will 
undergo  a  time-scale  transformation  which  is  frequently  referred  to  as 
a  Doppler  effect  or  Doppler  shift.  If  the  spatial  variation  is  an 
oscillation  or  is,  in  some  sense,  periodic  so  that  the  resultant  time- 
scale  transformation  exhibits  the  periodicity  described  in  the  hypothesis 
of  Theorem  (2-5),  and  if  the  original  process  is  WSS,  then  the  resultant 
Doppler-shifted  process  is  CS  (by  Theorem  (2-5)). 

This  Doppler  effect  is  encountered  frequently  in  studies  in  nuclear, 
atomic,  and  astronomical  physics,  and  in  radio-science,  radio-astronomy, 
and  other  fields  of  science. 

i)  Processes  derived  from  the  Poisson  counting  process  with  periodic  rate 
parameter:  In  contrast  to  the  synchronous  pulse-train  processes  such  as 
PAM,  one  often  encounters  processes  where  only  the  average  rate  of  pulse 
occurrences  is  known.  In  such  situations,  where  pulse  occurrences  are 


totally  asynchronous  (random),  the  Poisson  counting  process  is  frequently 
used  in  constructing  a  model  [2,3,22].  In  particular,  models  for  random 
facsimile  signals  [2],  random  telegraph  signals  [2,3],  and  shot  noise 
[2,3,22]  have  been  constructed  on  the  basis  of  the  Poisson  counting  process. 

If  the  rate  parameter  u  in  the  counting  process  is  constant,  then 
the  above  models  are  WSS.  However,  if  the  rate  parameter  varies 
periodically  with  time,  then  the  models  become  CS.  Such  CS  models  have 
been  proposed  in  studies  of  meteorology  [18]  and  noise  in  electronic 
devices  [22]. 

We  will  show  here  that  the  mean  and  autocorrelation  functions  for 

the  CS  random  facsimile  signal  derived  from  the  inhomogeneous  Poisson 

counting  process  with  rate  parameter  p(t)  =  uq  +  UjP(t)  are  identical  to 

those  corresponding  to  the  process  which  is  obtained  from  the  time-scale 

Vl  r 

transformation:  t'  =  t  +  —  Jp(t)  on  the  WSS  facsimile  signal  derived 

p0 

from  the  homogeneous  Poisson  counting  process  with  constant  rate  parameter 
(A  similar,  but  less  general  result,  for  the  random  telegraph  signal 
has  been  stated  by  Hurd  [5] . ) 

M0DEL(7):  Random  facsimile  signal.  A  random  facsimile  signal  is 
obtained  by  optically  scanning  a  black  and  white  picture  according  to, 
for  example,  the  line  scanning  scheme  discussed  in  the  following  sub¬ 
section.  For  a  random  picture,  the  scanner  output  x  will  be  a  two-valued 
random  process,  say  x(t)  =  1  for  black  portions  and  x(t)  =  0  for  white 
portions.  The  times  at  which  transitions  from  black-to-white  and  white- 
to-black  occur  will  be  an  ordered  sequence  of  random  variables  which  we 
will  model  in  terms  of  the  inhomogeneous  Poisson  counting  process  with 
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rate  paraaeter  v-  Using  precisely  the  same  assumptions  and  method  of 
derivation  as  Franks  [2],  but  generalized  from  the  case  of  a  constant 
rate  paraaeter  ji^  to  that  of  the  fluctuating  u(t),  we  obtain  the 
following  set  of  coupled  differential  equations:'0 


3P  (t,s) 

— - =  -M(s)[Pn(t,s)-Pnl(t,s)],  s  >  t,  n  >  0 

1,  n  =  0 


0,  n  /  0 


P  (t,s)  =  0,  n  <  0  , 
n 


(2-19) 


where  P^Ct.s)  is  defined  to  be  the  probability  that  exactly  n 
transitions  occur  in  the  interval  (t,s).  x  is  constant  over  each  interval 
defined  by  consecutive  transition  times,  and  assumes  the  values  1  or  0 
with  probabilities  p  or  1-p,  and  the  values  of  x  in  different  intervals 
are  assumed  to  be  statistically  independent. 

Now,  from  this  model,  we  find  that: 


mx(t)  =  E(x(t) }  =  Pr(x(t)=l)  =  p  V- t, 

k  (t,s)  =  E{x(t)x(s)}  =  Pr(x(t)=l  and  x(s)=l) 

XX 

=  p*Pr(t  and  s  are  in  the  same  interval) 

+  p2*Pr(t  and  s  are  in  diff?rent  intervals) 

=  p*PQ(t,s)  +  p2(l-PQ(t,s))  V- s  >  t, 

where  Pr(*)  indicates  "the  probability  of  the  event  in  the  parenthesis". 
But,  from  Eq.  (2-19),  we  easily  obtain  the  solution  for  P^: 

s 

PQ(t,s)  =  exp [-  /  jj (a)doj  ,  s  >  t  , 

10 See  also  Parzen  [22],  and  Papoulis  [3]. 
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so  that 


kxx*t,S^  =  <\  ’  ■JexPf"  /  **(o)do]  +  ■*  V-s  >  t, 

and  since  kxx(t,s)  is  symmetrical  in  t  and  s  then 

s 

kxxft,s)  =  (Bx  ■  "x)exPH  /  M(o)dof]  ♦  m*  V-  t,s. 

If  we  now  consider  a  process  y  with  constant  rate  parameter  p^. 


=  0\  "  «“x)exp[-yn|t-s|]  ♦  m*  , 


V-  t,s 


which  is  the  result  derived  by  Franks  [2] .  The  interesting  point  here 

is  that  if  we  subject  y  to  the  time-scale  transformation:  t'  =  t  +  —  /p(t), 

M0 

then  we  obtain  a  process  x'  with  the  same  mean  and  autocorrelation 
functions  as  the  above  process  x  with  fluctuating  rate  parameter 

p(t)  =  p0  +  MjP(t). 

We  conjecture  that  various  WSS  models  for  piecewise  constant  processes 

derived  from  the  Poisson  counting  process  with  constant  rate  parameter 

Up  can  be  extended  to  CS  models  with  the  time-scale  transformation: 

t'  =  t  +  —  f p(t),  where  the  resultant  mean  and  autocorrelation  functions 
U0 

will  be  identical  to  those  corresponding  to  the  model  obtained  directly 
in  terms  of  the  inhomogeneous  Poisson  counting  process  with  periodic  rate 
parameter:  p(t)  =  Pq  +  PjP(t). 

M0DEL(8):  Asynchronous  PAM.  A  specific  model  which  we  will  be  concerned 
with  in  Chapter  IV  can  be  derived,  via  a  time-scale  transformation,  from 
the  general  WSS  asynchronous  random  pulse-sequence 


y(t)  •  I  v0(t-tn) 


(2-20) 


where  (a)  is  an  arbitrary  WSS  sequence  independent  of  the  occurrence 
tines  {tn>  which  fora  an  ordered  sequence  distributed  according  to  the 
Poisson  counting  process  with  constant  rate  parameter  uQ,  and  qQ(t)  is 
an  arbitrary  pulse  shape.  This  process  y  has  been  used  as  a  model  for 
shot  noise  and,  as  shown  by  Franks  [2],  has  mean  and  autocorrelation 
functions  given  by 

00 

"y  =  Va  /  %(t)dt 

00 

k  (t)  =  m2  +  yAo2  /  q  (t+x)q  (t)dt, 

yy  y  o  a  J  o  o 

where  o2  is  the  variance  of  {a  }  and  m  is  the  mean.  Now,  we  create  a 
a  n  a 

new  process  x  by  subjecting  y  to  the  time-scale  transformation: 

t'  =  t  +  p(t),  where  p(t)  is  any  T-periodic  function  such  that  t'  (t)  is 

a  nondecreasing  function  (time  never  regresses),  and  obtain 

mx(t)  *  my 

kxx(t,s)  =  k  (t  *  p(t)  -  s  -  p(s)) 

as  the  mean  and  autocorrelation  functions  for  the  T-CS  asynchronous 
random  pulse-sequence  x  whose  average  rate  of  pulse- occurrences  (and  pulse- 
shapes)  varies  periodically  with  time. 
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3.  Multi -dimensional  Linear  Transformations  (Scanning) 

In  this  section  we  briefly  discuss  the  transformation  of  multi¬ 
dimensional  quasi-stationary  processes  into  one-dimensional  cyclo¬ 
stationary  processes,  and  the  application  of  this  concept  to  the  generation 
of  CS  processes  via  scanning. 

A  three-dimensional  process  y  is  a  random  function  of  three 
independent  variables:  y(t,u,v),  t  e  R,  u  e  Dj,  v  e  D^.  In  the  applications 
discussed  here,  u  and  v  represent  two  spatial  coordinates  and  t  the  usual 
time  coordinate.  The  domains  Dj  and  D2  will  be  finite  intervals  in  R  the 
set  of  real  numbers.  We  will  define  y  to  be  quasi -wide-sense  stationary  . 
if  and  only  if  each  of  the  one-dimensional  processes  y(*,u,v)  is  WSS  for 
each  value  of  u  e  Dj  and  v  e  . 11 

We  are  interested  in  linear  transformations  which  map  y  into  a  one¬ 
dimensional  CS  process  x  (a  random  function  of  the  single  variable  t). 

As  a  generalization  of  the  one-dimensional  linear  transformations 
discussed  in  Section  2  of  this  chapter,  we  characterize  a  three-dimensional 
linear  transformation  in  terms  of  a  three-dimensional  impulse  response  ;-*2 
If  y  is  the  input  to  a  transformation  G,  and  x  is  the  output,  then  the 
realizations  of  x  are  related  to  the  corresponding  realizations  of  y  as 
follows: 


x(t)  =  /  /  /  g(t,T,u,v)y(T,u,v)dTdvdu  ,  (2-21) 


llThis  notation  does  not  parallel  our  convention  for  one-dimensional 
processes,  but  is  the  least  ambiguous. 

12  The  terms  "three-dimensional  linear  transformation"  and  "three-dimensional 
impulse  response"  are  not  quite  appropriate  since  the  output  (image) 
process  is  one-dimensional. 


where  g(t,r,u,v)  is  the  three-dimensional  impulse  response. 

We  say  that  G  is  a  quasi -periodic  transformation  with  period  T  if 
and  only  if 

g(t+T,t+T,u,v)  =  g(t,x,u,v)  v-  t,T  e  R,  u  s  Dj,  v  e  D2  .  (2-22) 

With  these  definitions  we  can  generalize  Theorem  (2-4)  as  follows: 
THE0REM(2-6) :  If  the  input  y  to  a  three-dimensional  quasi-periodic 
transformation  with  period  T  is  a  three-dimensional  quasi-WSS  process, 
then  the  one-dimensional  output  x  is  a  T-CS  process. 

The  proof  directly  parallels  that  for  Theorem  (2-4). 

We  now  give  two  applications  of  Theorem  (2-6)  to  the  generation  of 
CS  processes  via  scanning: 

M0DEL(9):  Line-scanning  a  rectangular  field  (video).  The  process  of 

scanning  a  finite  rectangular  field--performed  by  a  video  camera,  for 

example--can  be  modeled  with  a  three-dimensional  linear  transformation. 

Then  the  scanner  is  fully  characterized  by  the  impulse  response  g(t,i,u,v) 

which  is  sometimes  referred  to  as  a  window  function.  With  reference  to 

Figure  (2-8)  and  Eq.  (2-21),  =  (0,W^)  and  D 2  =  (O.wp  are  the  domains 

for  u  and  v,  and  y(t  ,u  ,v  )  is  a  realization  of  the  three-dimensional 

0  0  0 

process  y  evaluated  at  time  tQ,  horizontal  distance  u  and  vertical 
distance  vq.  The  video  signal  is  the  scanner  output  process  x  and  is 
obtained  from  y  as  in  Eq.  (2-21). 

The  conventional  video  camera  employs  a  constant  velocity  line- 
scanner  whose  impulse  response  can  be  modeled  as  (with  some  abuse  of 
notation) : 


g(t,T,U,V)  =  g(Uj(t)  -  U,Vj (t)  -  V)6(t-t) 

where  the  functions  and  are  as  shown  in  Figure  (2-9),  and  each  are 
LT-periodic  where  T  is  the  time  required  to  scan  a  single  line  and  L  is 
the  number  of  lines  per  frame.  Therefore,  the  scanner  G  is  a  quasi- 
periodic  transformation  with  period  LT,  so  that  if  y  is  quasi-WSS  then 
the  video  process  x  is  CS  (period  LT),  by  Theorem  (2-6). 

If  the  above  window  function  g(*,*)  is  ideal--a  two-dimensional 
impulse  function- -then  we  have 

x(t)  =  y(t,u1(t),v1(t)). 

Using  this  ideal  model  for  the  scanner  along  with  the  assumption  of 
zero  retrace  time  (see  Fig.  (2-9)),  and  the  model  for  y  proposed  by 
Franks  [10],  results  in  an  autou.  .function  for  x  which  is 

composed  of  three  periodic  factors.  The  first  factor  characterizes  the 
correlation  within  any  given  line,  the  second  characterizes  line-to- 
line  correlation,  and  the  third  characterizes  frame-to-frame  correlation 
as  follows:. 


kxx(t.s)  =  k1(t-s)k2(t,s)k3(t,s) 


where,  for 


correlation  within  a  line: 


kj(t-s)  =  p} 


(t-s)modT 


■  1  - 


t-s-nT 


n=-° 


qQ(t-s-nT) 


(2-23 


(2-24 


and  is  T-periodic  in  (t-s)  (.qQ  is  a  rectangular  gate  function  of 
width  T) , 
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line-to-line  correlation: 

(t-tmodT)modLT  -  (s-smodT)modLT| 
k2(t,s)  =  p*  (2-25) 

•  l  p'n-B)"°dL  ,0(t-nT),o(S-»T) 
n,m=-» 

and  is  jointly  T-periodic  in  t  and  s, 
frame- to- frame  correlation: 

i|  (t-tmodLT)  -  (s-smodLT) | 

k3(t,s)  =  p*  (2-26) 

=  I  p|n"^  qj (t-nLT)q  (s-mLT) 
n,m 

and  is  jointly  LT-periodic  in  t  and  s  (qj  is  a  rectangular  gate 
function  of  width  LT) . 

Finally,  if  the  contribution  to  kxx(t,s)  due  to  frame-to-frame 
correlation  is  ignored  because  of  its  relatively  low-frequency  nature 
(this  is  done  in  practice  for  some  analyses  of  video  systems)  then  the 
period  of  cyclostationarity  reduces  from  LT  to  T  as  in  the  model 
introduced  in  Chapter  I. 

MODEL(IO):  Circular  scanning  (radar).  The  process  of  continuously 
scanning  a  one-dimensional  circular  field--performed  by  a  radar  antenna, 
for  example--can  be  modeled  with  a  two-dimensional  linear  transformation 
with  impulse  response  g(t,x,6).  With  reference  to  Figure  (2-10)  and  the 
two  dimensional  analog  of  Eq.  (2-21),  =  (0,2tt)  is  the  domain  for  6, 

and  y(t  ,0  )  is  a  realization  of  the  two-dimensional  random  process  y 
evaluated  at  time  tQ  and  angular  displacement  0  .  The  received  radar 
signal  is  the  scanner  output  process  x  and  is  obtained  from  y  as  in 
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Eq.  (2-21): 

x(t)  =  /  /  g(t,t,e)y(T,0)dedT  (2-27) 

R  Dj 

It  a  constant  velocity  scanner  is  used,  then  the  impulse  response 
can  be  modeled  as  (with  some  abuse  of  notation) : 


g(t,T,0)  =  g(0j(tj  -  0)<S(t-r) 


where  the  function  is  as  shown  in  Figure  (2-11; ,  and  is  T-periodic, 
where  T  is  the  time  required  for  one  complete  revolution  of  the  scanning 
antenna.  Therefore  G  is  a  quasi-periodi c  transformation  with  per  M  T, 
so  that  if  y  is  quasi-WSS  then  the  radar  process  x  is  T-CS  by  th .  two- 
dimensional  version  of  Theorem  (2-6). 

As  an  example  of  a  "non-ideal"  window  function  g,  consider  the 
rectangular  window  shown  in  Figure  (2-12).  For  this  case,  the  signal 
x  is  given  by 


2tt  “ 

x(f>  =  /  /  g(Mt)  -  0)S(t-T)y(T,e)did0 

0  -00  1 
2  7T 

=  /  g(e.(t)  -  e)y(t,e)de 

:  a-to 


/ 


Mt)-* 


y  (t ,  9)d0 


1 


0 


2rrt  n_ 

5  f  T  +  10  ,, 

-  J  yit,6)uo 

2nt  it 

T  "  \0 


and  is  the  average  value  of  y(t,0)  over  the  range  (-(t/10,tt/I0)  of  9. 

In  Section  6,  we  generalize  *he  discussion  here  to  random  multi¬ 
dimensional  linear  transformations,  and  show  that  random  scanners  with 
random  window  shapes  and  velocities  can  also  generate  cyclostationary  signal 


4.  Nonlinear  Transformations 


Many  nonlinear  transformations  which  arise  in  engineering  problems 
can  be  characterized  with  Wiener's  Vol terra  series  representation  {44-47] 
in  which  the  output  x(t)  of  an  N^-  order  Volterra  system  driven  by  the 
input  y(t)  is  expressed  as 

N  OO  00 

X(t)  =  l  /•••/  gn(t-T1,t-T2>“,t-Tn)y(T1)**y(Tn)dT1**dTn, 
n=l-°°  -00 

where  the  kernels  {g^}  can  be  thought  of  as  generalized  impulse  response 
functions.  This  series  is  sometimes  referred  to  as  a  power  series  with 
memory . 

Note  that  a  first  order  Volterra  system  (N  =  1)  is  simply  a  linear 
system  and  the  above  representation  is,  for  this  case,  the  super-position 
integral  (Eq.(2-1))  used  in  Section  2  for  representing  linear  transformations. 
Hence  gj  is  for  this  case,  the  usual  impulse-response  function. 

The  nonlinear  transformations  of  interest  in  this  thesis  can,  for 
the  most  part,  be  characterized  with  a  generalized  Volterra  series  in 
which  the  kernels  are  invariant  under  joint  shifts,  in  their  arguments, 
which  are  multiples  of  a  basic  period  T: 


gn(-t,Trt,T2;’*’’t,Tn^  =  8n(t+T,T1+T;**st+T,xn+T),  V- n 


for  all  t, 

whereas,  the  kernels  in  the  standard  Volterra  series  are  invariant  under 
arbitrary  joint  shifts. 

In  addition  to  allowing  periodically  time-varying  kernels,  we  will 
also  include  a  T-periodic  zero  th  order  term  so  that  our  generalized 
periodic  volterra  series  becomes:  v 
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!x(t)  =  gQ(t)+  £  /••/gn(t,T1;**;t,Tn)y(T1)**y(Tn)dT1»*dTn  .(2-28) 

n=l  i  - 

1 

We  will  refer'  to  this  more  general  class  of  Volterra  systems  as 

i  *  i  < 

periodic  Volterra. systems.  For  this  class,  we  have  the  following 

'  f  * 

generalization  of  Theorem  (2-4)  on  the  generation  of  CS. processes: 

.  *  ,  l 

THEOREM (2 -7)  3  The  output  process  x  of  an  Nr^-  order  T-periodic  Volterra 

S*  '  i 

system,  which  is  driven  by  a  stationary  -  of -order  2N  (S(2N))  input 


process  y,  is  T-CS. 


Proof : 


i  i 


i)  From  Eq.  (2-28),  the  mean  of  the  Output  x  is  .  .  y 

•  5  I 

N  '* 

mx(t)  =  gQ(t)+  I  ./•••/gn(t,T1;**;t,T^)E{y(x  )*»y(t  j'JdTj'-dr  • 
n=l  .  ’  i 

N  ,  '  , 

=  g0(t+T^+  I  /*  */g^(t+T»  tj+T; • • ; t+T,tn+T) • 


E{y(T1)***y(in)}dx1“*dTn  . 


=  g0(t+T)+  l  /•'•/gn(t^T,T';-st+T,i'). 

n=l  1  1  ' 

I  4 

’  1 

E{y It! -T) • • *y (t' -T) }dx{ * • *dx' 

7  1  n  1  n 

N 

=  gQ(t+T)+  l  /•  '/gn  (t+T,  x| ;  •  •  ;t+T,  t^)E{y  (x|)  •  *y (x^)  }dx|  *  'dx^ 
n=l  '  !  :  i 


=  mx(t+T) 


V-  t.  " 


where  the  next-to-last  equality  is  a  result  of  the  S(2N)  propertyof  y. 


13A  process  is  S(N)  iff  ’  cs  1—  through  N—  order  joint  PDF's  are  invariant 
under  arbitrary  joint  shifts  in  their  indexing  arguments  [3].  , 


ii)  Again,  using  Eq.  (2-28),  we  have 

kxx(t’iO  =  g0(t)g0(s)  +  go(t)mx(s)  +  gQ(s)mx(t) 

N  N 
n=l  m=l 

E{y  (Tj)  •  *y  (Tn)y  (Yj)  •  *y  (Ym)  >dXj  •  ‘dx^Yj  *  *dY 

Using  the  periodicity  of  the  (gn>  and  of  mx>  and  the  stationarity  of  y, 
as  in  part  i),  yields  the  result: 

kxx(t,s)  =  kxx(t+T’S+T>  V-t,s. 

Hence,  from  i)  and  ii),  x  is  T-CS. 

QED 

MODEL(ll):  Parametric  amplifiers.  The  parametric  amplifier  [48]  is  an 
example  of  a  nonlinear  system  which  can  be  accurately  modeled  as  a 
periodically  time-varying  nonlinear  transformation  with  memory,  and  might 
therefore  be  modeled  as  a  periodic  Volterra  system.  Hence,  the  output 
of  a  parametric  amplifier  driven  by  a  strict-sense-stationary  (SSS)  process 
might  well  be  expected  to  be  CS,  and  has,  in  fact,  been  so  modeled  in 
Reference  [21]. 

In  the  next  two  subsections,  we  consider  special  periodic  Volterra 
systems  which  generate  CS  process  from  input  processes  which  are  only  S(2). 

a)  Zero-memory  nonlinearities.  If  the  kernel  functions  in  the 
periodic  Volterra  series  representation  of  a  system  take  the  form 

gn(t,T1;t,x2;-**;t,Tn)  =  gn(t)6(t-x1)**«6(t-xn) 


14A  process  is  SSS  iff  it  is  S(N)  for  all  natural  numbers  N  [3]. 
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where 

then  we  have  the  reduced  input-output  relation 
N 

x(t)  =  l  g  (t)(y(t))n, 
n=l 

and  the  system  is  a  zero -memory  periodic  nonlinearity.  The  input-output 
relation  for  these  systems  can  be  compactly  expressed  as 

x(t)  =  G(y(t),t) 

where  G(*,t)  is  the  nonlinear  function  with  power  series  coefficients 
and 

G(-,t+T)  =  G(-,t)  V- t. 

For  these  systems,  the  output  x  at  any  time,  say  tQ,  depends  on  the  input 
y  only  at  time  tQ,  and  on  the  time  tQ  itself. 

For  this  class  of  zero-memory  nonlinear  systems,  we  have  the 
following  theorem  on  the  generation  of  CS  processes: 

THE0REM(2-8) :  If  the  input  y  to  a  zero-memory  time-varying  nonlinearity, 
which  satisfies  the  periodicity  condition:  G(',t)  =  G(‘,t+T)  for  all  t, 
is  S(2)  then  the  output  x  is  T-CS. 

Proof : 

00 

m  (t)  =  E{G(y(t) ,t) }  =  /  G(o,t)p(o)do 

-oo 

00 

=  /  G(o,t+T)p(a)da 

-  CO 

-  m  (t+T)  V-  t  > 

X 


where-  p(*)  is  the  PDF  for  y(*)  for  all  t;  and 


kjcx^’S)  =  E<G(y(t),t)G(y(S),S)> 

« 

=  //  G(o,t)G(Y,s)pts(o,Y)dodY 
•  00 
CO 

=  //  G(o,t+T)G(Y,s+T)p^.4 fTj_(s+T) (°»Y)dodY 
=  (t+T ,  s+T)  H,s, 

where  p^  s(*»‘)  is  the  joint  PDF  for  y(t)  and  y(s).  Hence  x  is  T-CS. 

QED 

As  an  example  of  a  transformation  which  satisfies  the  hypothesis 
of  Theorem  (2-8),  consider  the  form: 

G(y(t),t)  =  G(y  (t)  +  q(t)) 

where  q(t)  is  any  T-periodic  function.  Here,  the  signal  y  is  subjected 
to  an  additive  periodic  component  q(t)  and  then  applied  to  a  nonlinear 
device  such  as  a  diode,  square- law  device,  or  some  type  of  threshold 
device.  This  type  of  transformation  is  frequently  incurred  in  optimum 
detectors  for  periodic  signals  in  stationary  noise.  Note  that  if  G  were 
a  square-law  device,  then  the  generalized  Volterra  kernels  would  be  given 
explicitly  by 

gn  =  0  n  >  3 

g2(t,T1;t,T2)  =  5(t-r1)6(t-T2) 

=  2q(t)6(t-t1) 
gQ(t)  =  q2(t)  . 
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b)  Nonlinear  Modulation  of  synchronous  pulse -trains.  If  the  kernel 
functions  in  the  periodic  Vol terra  series  representation  take  the  fora 

00 

gn(t,Tl;”;t*Tn)  =  J  %  (t-iT)  6  (iT-tj  )  •  •  •  6  (iT-tn) , 

i=-o° 

then  we  have  the  reduced  input-output  relation 
®  N 

x(t)  =  l  [  l  gn(t-iT)(y(iT))n] 
i=-a>  n=l 

which  can  be  expressed  in  the  more  compact  form 
00 

x(t)  =  l  q(t-iT,y(iT)), 

i=-« 

where  q(t,*)  is  the  nonlinear  function  with  power  series  coefficients 
(gn(t)}.  This  last  expression  represents  a  signal  format  which  has 
received  much  attention  as  a  means  for  transmitting  random  sequences 
of  numbers  {a^}  =  {y(iT)},  and  is  referred  to  as  a  modulated  synchronous 

pulse-train. 

With  this  signal  format,  each  pulse  q(t-iT,*)  carries  the  realization 
of  one  random  number,  a^.  If  the  modulation  is  linear,  then  the  signal  is 
PAM,  as  discussed  in  Section  2b,  and  the  random  numbers  are  carried 
as  pulse-amplitudes.  However,  if  the  modulation  is  nonlinear,  then  there 
is  a  multitude  of  different  schemes--each  with  its  own  theoretical  and 
practical  advantages  and  disadvantages.  Following  are  six  specific 
examples : 

M0DEL(12):  Frequency-shift-keying  (FSK) .  A  sinusoidal  signal  with 
gating  envelope  w(t)  is  frequency-shifted  by  the  parameters  { a^ }  such  that 

q(t-nT,an)  =  w(t-nT)cos(2Ti(f+an)  (t-nT)) 

=  w(t-nT)cos  (2ir(f+an)t) 
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where  the  second  equality  is  valid  if  (f+an)T  is  an  integer  for  every 
realization  of  a^  for  every  n, 

M0DEL(13):  Phase-shift-keying  (PSK).  A  sinusoidal  signal  with  gating 
envelope  w(t)  is  phase-shifted  by  the  parameters  {a^}  such  that 

q(t-nT,an)  =  w(t-nT)cos(2irf  (t-nT)+an) 

=  w(t-nT)cos(2Trft+an) 

where  the  second  equality  is  valid  if  fT  is  an  integer. 

M0DEL(14):  Pulse-width-modulation  (PWM).  The  width  (duration)  of  a 
basic  pulse  shape  qQ(t)  is  determined  by  the  parameters  {a^}  such  that 

q(t-nT,an)  =  qQ((t-nT)/an) 

where  {an>  take  on  only  positive  values. 

M0DEL(15):  Pulse-position-modulation  (PPM).  The  epoch  (time-of- 
occurrtnce)  of  a  basic  pulse  qQ(t)  is  determined  by  the  parameters  {a^} 
such  that 

q(t-nT,a  )  =  q  (t-nT-a  ) 

where  {a^}  take  on  only  values  which  are  non-negative  and  less  than  T. 

M0DEL(16):  Digital  pulse-frequency-modulation  (DPFM) .  The  frequency-ot- 

occurrence- -number- -of  basic  pulses  qQ(t)  in  each  time-slot  is  determined 

by  the  parameters  {a^}  such  that 

a^ 

q(t-nT,an)  =  l  qo((t-nT)an-(i-l)T), 
i=l 
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where  tan>  take  on  only  non-negative  integer  values.  Note  that  if  qQ(t) 
is  an  integral  nuaber  of  cycles  of  a  sinusoid  with  duration  T,  then  this 
DPFM  signal  is  an  FSK  signal  with  rectangular  envelope  w(t). 

M0DEL(17):  General  M-ary  signaling.  The  parameters  {aR}  serve  to  index 
a  set  of  M  basic  pulse-shapes,  which  need  not  be  "functionally"  related 
as  in  the  above  cases,  such  that 

q(t-nT,an)  =  qa  (t-nT) 
n 

where,  for  example,  (q..(t);  i  =  1,2,3,...,M}  might  be  a  set  of  mutually 
orthogonal  pulses  duration  limited  to  the  interval  (0,T) . 

Paralleling  Theorem  (2-8),  we  have  the  following  theorem  which 
establishes  a  sufficient  condition  for  the  general  synchronous  pulse- 
train  signal  to  be  CS: 

THEOREM(2-9) :  If  x  is  a  synchronous  pulse-train  with  period  T  and  the 
modulating  sequence  {a^}  is  stationary -of -order -two,  then  x  is  T-CS. 
Proof : 

x(t)  =  1  q(t-nT,a  )  so  that 
n 

00 

mx(t)  “  /  l  q(t-nT,o)p(o)do 
A  -00  n 
00 

»  /  l  q(t+T-(n+l)T,o)p(o)do 

-oo  n 
00 

*  /  l  q(t+T-n'T,0)p(a)da 
-00  n» 

=  mx(t+T)  vt, 

whei"e  p(‘)  is  the  PDF  for  a^  for  every  n,  and 


kxx^t,s^  =  //  I  q(t-nT,a)q(s-mT,Y)p  (.o,y)dady 

n,m  n'ffl 

00 

~  II  1  q(t+T“(n+l)T,cr)q(s+T-(m+l)T,Y)p^n+1j _ (m+j) (°>Y)dodY 
00 

=  //  l  q (t+T-n ' T , o)q (s+T-m' T, Y)p  ,  m,(a,Y}dadY 
-«»  n*  ,m'  n  "m 

=  kxx(t+T,s+T)  V-t,s, 

where  pt  s(*,0  is  the  joint  PDF  for  x(t)  and  x(s).  Hence  x  is  T-CS. 

QED 

In  Section  7  we  present  a  generalized  model  for  pulse-train  signals 
which  incorporates  random  jitter  wherein  the  pulse  train  is  no  longer 
synchronous  but  the  signal  is  still  CS. 
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5.  Random  Linear  Transformations 

The  developments  in  this  section  on  random  linear  transformations 
closely  parallel  those  in  Section  2  on  deterministic  linear 
transformations.  Due  to  the  length  of  this  section,  we  begin  by  giving 
a  brief  outline  of  the  theorems  proved  and  the  cyclostationary  process 
models  defined: 

a)  Wide-sense-stationary  systems 

Theorem (2- 1 0) :  Preservation  of  wide-sense  stationarity  by  random 
transformations . 

Model (18):  Random  periodic  signals 

Theorem(2-ll) :  Conversion  of  deterministic  periodic  systems  into  wide- 
sense-stationary  systems  via  phase-randomization. 

Theorem (2-12) :  Preservation  of  cyclostationarity  by  random  transformations. 

b)  Cyclostationary  systems 

Theorems (2-13) :  Generation  of  cyclostationarv  processes  from  stationary 
(2-14) 

processes  via  random  transformations. 

Model  (19):  Random  periodic  attenuation  of  stationary  processes 
Model  (20) :  Multiplication  of  a  stationary  process  by  a  cyclostationary 
process 

c)  Random  time-scale  transformations 
i)  Phase  randomization: 

Theorem(2-15) :  Reduction  of  cyclostationary  processes  to  stationary 
processes  via  phase-randomization 

Theorem(2- 16) :  equivalence  between  phase-randomizing  and  time-averaging 
of  random  processes. 

Theorem(2-17) :  Preservation  of  system-cyclostationarity  by  phase- 


randomization. 


67 


Theorem(2-18) :  Reduction  of  cyclostationary  systems  to  wide-sense¬ 
stationary  systems  via  phase-randomization. 

Theorem (2- 19) :  Equivalence  between  phase-randomizing  and  time-averaging 
random  systems. 


Theorem (2- 20) :  Preservation  of  cyclostationarity  by  a  stationary  time¬ 


scale  transformation. 


Theorem(2-21) :  Generation  of  cyclostationary  processes  from  periodic  signals 
via  stationary  time-scale  transformations. 

Model (21):  Phase-modulated  signal 

Model (22):  Frequency-modulated  signal. 

Theorem(2-22) :  Generation  of  cyclostationary  processes  from  PAM  time-scale 
transformations . 

Model (23):  Digital  phase-modulation 

Model (24):  Digital  frequency-modulation 

Theorem (2-23) :  Generation  of  cyclostationary  processes  from  stationary 
processes  via  cyclostationary  time-scale  transformations. 
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A  random  transformation,  like  any  other  random  process,  can  be 
thought  of  as  an  ensemble  of  deterministic  transformations.  A  single 
deterministic  input  to  a  random  transformation  yields  an  ensemble  of 
deterministic  outputs  (a  random  output  process)  with  each  output  resulting 
from  one  member  of  the  ensemble  of  transformations.  Of  course,  for  any 
given  event,  only  one  member  of  the  ensemble  of  transformations  will 
occur  and  a  single  input  will  result  in  a  single  output.  In  this  section 
we  will  be  primarily  concerned  with  random  transformations  of  random 
processes.  In  particular,  we  will  consider  linear  wide-sense-stationary 
and  wide- sense-cyclostationary  transformations  (defined  herein)  in 
connection  with  the  generation  and  preservation  of  cyclostationarity. 

All  random  linear  transformations  of  interest  in  this  thesis  will 
be  integral  transformations  characterized  by  random  impulse-response 
functions.  Thus,  input  and  output  processes  will  be  related  as  follows 
00 

x(t)  =  /  g(t,T)y(T)dr  (2-29) 

-00 

where  y(x)  is  a  realization  of  the  input  process,  g(t,T)  is  a  realization 
of  the  two-dimensional  random  impulse-response  process,  and  x(t)  is  the 
resultant  realization  of  the  output  process.  We  will  consider  only  those 
random  transformations  which  are  statistically  independent  of  their 
inputs . 

As  pointed  out  by  Zadeh  [37],  the  system  function (defined  in  Sec.  2) 
can  be  useful  for  characterizing  random  linear  systems  (transformations) 
as  well  as  deterministic  linear  systems.  Using  the  same  notation  as  for 
deterministic  systems,  we  denote  the  r.  dom  system  function  for  a  random 
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system  G  as 

00  , 

H(t,f)  =  /  h(tfT)e“j27rfTdT 

_oo 

=  /  g(t,t-T)e":’27rfTdT. 

.00 

In  addition,  we  denote  the  autocorrelation  function  for  the  random 
system  function  as  Kg  and  refer  to  it  as  the  system  autocorrelation 
function  for  G: 


Kg(t,s,f ,v)  -  E{H(t,f)H*(s,v)} 

00 

=  //  E{h(t,T)h*(s,Y)}e"j27r(fT‘VY)dTdY.  (2-30) 

-  CO 

We  note  that  Kg  is  a  generalization5 of  the  quantity  referred  to  as  a 
system  correlation  functiori  by  Zadeh  [37] . 

We  proceed  to  derive  an  input -output  relation  for  autocorrelation 
functions  in  terms  of  Kg.  From  Eq.  (2-29)  we  have: 

00 

x (t)x* (s)  =  //  g(t,T)g*(s,y)y(T)y*(Y)dTdy 

-  OO 

00 

=  //  h(t,t-T)h*(s,s-Y)y(T)y*(Y)dTdy 

-  00 

00 

=  //  h(t,x)h* (s,y)y (t-x)y* (s-y)dTdy  . 

-  00 

Equating  the  expected  values  of  both  sides  of  the  above  equations  yields: 


k  (t,s)  =  //  E{h(t,t-x)h*(s,s-y)}k  (x.y)dxdy 

-  00  '' 


//  E{h(t,x)h*(s,y)}k  (t-x.s-y)dxdy. 

.00  *  • 


(2-31) 


15  K„(t,s,f,v)  is  also  the  inverse  double  Fourier  transform  w.r.t.  f'  and  v' 

of  Ig(f ' ,v' ,f ,v)  which  is  referred  to  as  the  bifrequency  intensity  spectrum 

and  is  the  autocorrelation  function  for  the  bifrequency  function  for  the 
system  G  (see,  for  example,  Middleton  [41]). 
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Using  Fourier  transforms  in  a  manner  similar  to  that  used  to  derive 
Eq,  (2-5)  results  in  tne  equation: 

00 

k  ft.s)  =  //  K„(t,s,f,v)k  lf,v)ej2,T(ft*vs)dfdv.  (2-32) 

w  _co  vi  yy 

Equations  (2-30)- (2-32)  will  piove  useful  in  the  following  development-. 

a)  Linear  wide-sense-stationary  systems.  An  interesting  subclass  of 
random  linear  transformations  are  those  which  preserve  wide-sense- 
stationarity:  the  output  of  a  member  of  this  class  will  be  WSS  if  the 
input  is  WSS.  Although  the  most  general  class  of  deterministic  linear 
systems  which  preserves  wide-sense-stationarity  is  the  class  of  time- 
invariant  filters,  it  is  not  true  that  the  most  general  class  of  random 
linear  -ystems  which  preserves  wide-sense-stationarity  is  the  class  of 
random  systems  with  ensemble  members  all  of  which  are  time- invariant  filters. 
The  following  theorem--first  stated  and  proved  by  Zadeh  [37]  for  ergodic16 
random  processes  and  transformations--establishes  a  general  class  of 
random  systems--to  be  called  wide-sense-stationary  (WSS)  systems-- 
which  preserves  wide-sensestationarity; 

THEOREM (2- 10) :  If  the  system  autocorrelation  function  for  a  random 
system  G,  whose  input  y  i\  a  zero-mean  WSS  process,  satisfies  the 
stationarity  condition: 

KG(t,s,f,f)  KG(t-s,0,f,f) 

for  all  t,s,f  then  tne  output  x  is  a  zero-m^an  WbS  process  . 


16Zadeh  used  time  averages  rather  than  ensemble  averages  in  nis  paoer. 
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Proof : 

i)  From  Eq.  (2-29),  the  mean  of  the  output  x  is 

00 

m  (t)  =  /  E{g(t,t)}m  (x)dx 

-00  * 

=  0  V-  t  since  m  (xj  =  0  V-  x. 

y 

ii)  From  Eq.  (2-32),  we  have 

00 

k  (t,s)  =  JJ  K  (t,s,f,v)K  (f)6(f-v)ej2lT(ft"Vs)dfdv 

— oo  y  y 

00 

=  /  K  (t-s,0,f,f)K  (f)ej27rf(t's)df 
-oo  y  y 

=  kxx(t-s)  V-  t,s  (with  some  abuse  of  notation) 

Hence,  from  ij,  ii)  x  is  WSS  . 

QED 

Notice  that  if  all  realizations  of  the  random  system  G  are  time- 
invariant  filters,  then 

KG(t,s,f,f)  =  E{H(t,f)H*(s,f) } 

=  E{H(0,f)H*(0,f) } 

-  KG(0,0,f,f)  V-t,5,f 

so  that  G  is  clearly  a  WSS  system. 

Following  are  four  specific  examples  of  WSS  systems  which  have 

been  used  in  analyses  dealing  with  statistical  communication  theory17  [38] . 

Notice  that  the  realizations  of  the  first  WSS  filter  presented  are  time- 

17A  number  of  more  sophisticated  models  for  WSS  systems  (communication 
channels)  are  defined  and  discussed  at  length  in  the  very  "readable"  report 
of  Maurer  [38].  Also,  a  number  of  models  (for  physical  and  biological 
applications  in  oceanography)  for  random  time-varying  channels  are  derived 
in  the  extensive  four-part  paper  of  Middleton  [42]. 
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varying  systems  (multipliers):  ,  '  i  , 

(1)  Random  WSS  multipliers:  If  an  input  process  y  is  multiplied  by  a 
WSS  process  z  to  yield  an  output  process  x,  then  realizations  of  the 

I  , 

random  impulse-response  function  which  characterizes  this  system  take 
the  form 

'  I 

g(t,T)  =  Z(t)Srt-T) 

and  h(t,t)  =  g(t,t-T)  =  z(t)6(r)  1 

•-  •'  J 

t  '  , 

I  ■  : 

The  system  autocorrelation  function  is,  from  Eq.  (2-30), 

i 

Kg(t,s,f , v)  =  //  E{z(t)6(i)z* (s)6(y))e  ^2lI^T  ^^dtdy 

-00 

'=  E{z(t)z*(s)}  .  1 

s  ! 

=  kZ2(t'S:>  *t,S,f,V' 

so  that  K,.(t,s,f,f)  -  K  (t-s,0,f,f)  for  all  t,s,f  and  G  is  a  WSS  system, 
b  b  v 

(2)  Random  first-order  filter:  The  random  first-order  .'ilter  i,s  simply 

a  deterministic  first-order  filter  with  its  inverse-time-constant  replaced 
with  a  random  variable  cj  so  that  we  have 

h(t, t)  =  e  u(t) 

where  u(*)  is  the  unit  step  function.  Thus,  if  p  (♦)  is  the  PDF  for  w, 
then  the  inverse  double  Fourier  transform  of  the  system  autocorrelation 

I  '  i 

function  K„  is 
b 

00 

E{h (t ,  t)h*  (s ,y) }  =  /  e  ‘71,|'T+Y'jpw(a)dou(T)u(Y) 


conjugate  characteristic  function  for  A. 
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The  following  class  of  CS  signals  are  generated  by  the  class  of 
random  time-invariant  filters: 

M0DEL(18):  Random  periodic  signals.  In  some  biomedical  experiments,  a 
subject  modeled  as  a  random  time- invariant  linear  system  is  excited  by 
a  known  deterministic  periodic  signal.  The  measured  response  signal  is 
then  modeled  as  a  random  periodic  signal:  period  known,  but  harmonic 
content  random.  In  addition  to  these  biomedical  test  signals,  the  periodic 
timing  and  test  signals  sent  through  communication  transmission  channels 
can  also  be  modeled  (at  the  receiver)  as  random  periodic  signals.  All 
signals  in  this  class  are  CS  and  take  the  form 

xtt)  -  l  anei2™t/T 

where  {a  }  is  a  random  sequence  with  the  constraint  a  =  a*  V-  n. 

It  is  apparent  from  the  discussion  and  examples  in  Section  2  of 
this  chapter  that  periodically  varying  systems  are  very  common  components 
in  communication  systems.  Although  a  suitable  deterministic  model  for  a 
periodic  system  may  well  be  known,  it  is  not  uncommon  for  the  phase 
(time-origin)  of  the  periodic  variations  to  be  unknown.  In  such  cases  it 
is  sometimes  sufficient  to  model  the  phase  as  a  random  variable  uniformly 
distributed  over  one  period.  In  fact,  in  some  analyses,  this  is  done 
even  though  the  phase  is  known  (this  point  will  be  discussed  in  Chapter  IV). 
The  following  theorem  establishes  the  fact  that  periodic  systems  with 
uniformly  distributed  random  phases  are  WSS  systems: 

THE0REM(2-11) :  If  the  phase  (time-origin)  of  a  deterministic  T-periodic 
system  is  modeled  as  a  random  variable  0  uniformly  distributed  over  one 
period,  say  (-T/2,T/2),  then  the  resultant  random  system  is  WSS. 
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Proof:  The  PDF  for  the  uniformly  distributed  random  variable  8  is 
;  1/T,  j  o |  <  T/2 

Pfl(<0  =  1  ,  .  ' 

{0  |rj|  >  T/2 

Thus , 

KG(t,s,f,v)  =  E{H(t+0 ,f)H*(s+0 , v) } 


=  /  H(t+o,f)H*(s+a, v)PQ(a)da 

.00 

,  t/2 

=  -  /  H(t+a,f)H*(s+a,v)dcr 
-T/2 

,  s+T/2 

=  i  /  H(t-s+Y,f )H* (y,f )dy 

s-T/2 

But  since  H(t,f)  is  T-periodic  in  t,  then  the  integrand  is  T-periodic 
in  y  and  the  integral  is  simply  an  average  over  one  period  and  is 
therefore  independent  of  the  absolute  end  points  of  the  one-period  interval, 
so  that 


,  T/2 

Kr(t,S,f,v)  =  i  /  lI(t-s+Y, f )H* (Y,f )dy 
b  -T/2 

00 

=  /  H(t-s+Y»f)H’"(Y,f)P0(Y)dY 

-  00 


=  K„(t-s,0,f,v)  ¥■  t,s,f,v  . 

b 

Hence  by  Theorem  (2-10),  G  is  a  WSS  system. 


QED 

i)  Preservation  of  cyclostationarlty.  While  it  is  true  that  all  WSS 
systems  preserve  wide-sense-stationarity  (by  definit.on),  it  is  not  true-- 
as  might  be  expected--that  all  WSS  systems  preserve  cyclostationarity . 
However,  all  the  WSS  system  examples  considered  above  do,  indeed,  preserve 
cyclostationarity.  In  fact,  most  WSS  systems  of  practical  interest  do 
preserve  cyclostationarity.  This  next  theorem  defines  the  most  general 
subclass  of  WSS  systems  which  preserve  cyclostationarity: 


THEOREM (2-1 2) :  If  the  system  autocorrelation  function  for  a  random 
system  G,  whose  input  y  is  a  zero-mean  T-CS  process,  satisfies  the 
stationarity  condition: 
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KG(t,s,f,v) 


K(t-s,e 


j2ir(ft-vs) 


,f»v) 


for  all  t,s,f,v  and  for  some  function  K,  then  the  output  process  x  is  a 
zero-mean  T-CS  process. 

Proof:  The  mean  of  x  is  (as  in  the  proof  of  Theorem  (2-10))  obviously 
zero,  and  from  Eq.  (2-32)  along  with  the  stationarity  condition  we  have 


kxx(t's) 


Now,  taking  advantage  of  the  periodicity  of  k  ,  we  can  expand  its 
double  Fourier  transform  K  into  a  series  of  the  form  of  Eq.  (2-14): 

00 

K  (f.v)  =  l  Kn(f)«(f-v+n/T)  V-  f,v. 


n=-° 


so  that 


k  (t,s)  =  //  K(t-s,ej27l(ft'vs),f,v)£  K  (f)6(f-v*n/T) 

AA  It 

-00  n 

ej2x(ft-vs)dfdu 

.  I  f  K(t-s,ej2,,(f(t-s)-"s/T) 


n 


,f,f-n/T)Kn(f)- 


J2ir(f  (t-s)-ns/T)  ,, 


Since  this  last  expression  is  jointly  T-periodic  in  t  and  s,  then  x  is 
a  zero- mean  T-CS  process. 


QED 
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An  example  of  a  random  system  which  satisfies  the  "stationarity 

condition"  of  both  Theorems  (2-10)  and  (2-12),  but  does  not  satisfy  the 

more  stringent  and  more  obvious  condition:  K~(t,s,f,v)  =  K,,(t-s,0,f ,v) 

b  b 

for  all  t,s,f,v — which  is  satisfied  by  all  previous  examples--is  the 
random  S (2)  delay.  This  random  system  has  random  impulse  response 

g(t,t)  =  6(t-z(t)) 

where  z  is  any  S (2)  process,  and  its  system  function  is 

so  that  its  system  autocorrelation  function  is 

=  f  e)2»(fo-VY)p  (0,y)dljdie-j2'(ft-«) 

.00 

t-s 

where  s(f,v)  is  the  double  Fourier  transform  w.r.t.  a  and  y  of 
Pt  S(°>Y)  which  is  the  joint  PDF  for  z(t)  and  z(s).  The  realizations  of 
the  T-CS  output  x  which  result  from  any  T-CS  input  y  take  the  form 

x  (t )  =  y (z (t) ) . 

Since  z  is  S(2),  then  the  time  parameter  for  y  does  not  increase 
monotonicaliy  and,  in  fact,  "hovers"  about  the  constant  mean  value  of  z, 
so  that  G  is  a  rather  poculiar  transformation.- 

b)  Linear  cyclostationary  systems.  In  the  last  subsection  we 
defined  the  class  of  WSS  systems  to  be  those  which,  when  driven  with  a 
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WSS  input  process,  produce  a  WSS  output  process.  We  now  define  the  class 
of  cyclostationary  (CS)  systems  to  be  those  which,  when  driven  with  a 
WSS  input,  produce  a  CS  output. 

THEOREM (2- 13):  If  the  system  autocorrelation  function  for  a  random 
system  G,  whose  input  y  is  a  zero-mean  WSS  process,  satisfies  the 
cyclostationarity  condition: 

r 

K„(t,s,f,f)  =  K_(t+T,s+T,f,f) 
b  b 


for  all  t,s,f  and  for  some  non-zero  T,  then  the  output  process  x  is  a 
zero-mean  T-CS  process . 

Proof:  From  Eq.  (2-32)  we  have 


kxx(t-s) 


*  II  K  (f)6(f-v)eJ2”(ft'v5)dfdv 

-00  ** 

00 

=  /  K  (t,s,f,f)K  (f)ej2lTf(t"s)df 

-00  *  ' 

=  /  KG(t+T,s+T,f,f)Kvv(f)ej2lTf((t+T'‘(s+T))df 


yy 


=  k  (t+T,s+T)  ¥  t,s, 

AA 


and  the  mean  for  x  is  clearly  zero.  Hence,  x  is  T-CS. 


QED 

A  large  subclass  of  CS  systems  is  the  class  of  random  periodic 
systems.  The  members  of  this  class  have  ensembles  with  elements  all  of 
which  are  periodic  deterministic  systems.  These  random  periodic  systems 
can  be  characterized  by  a  random  system  function  H  which  is  a  one¬ 
dimensional  random  process  indexed  by  f  with  parameter  t,  such  that  the 
random  processes  H(t+T,*)  and  H(t,*)  are  identical  for  all  t. 


Such  random  system  functions  can  be  represented  by  a  random  Fourier  series 

H(t,*)  =  l  H  (•)e]2imt/T 
n=-« 

where  the  coefficients  are  random  functions.  Thus,  paralleling 

deterministic  periodic  systems,  we  see  that  random  periodic  systems  have 

realizations  with  the  structure  shown  in  Figure  (2-2)  where  the  path- 

filter  transfer  functions  are  realizations  of  the  random  coefficient 

functions  H  . 

n 

The  fact  that  random  periodic  systems  are  CS  systems  is  obvious,  but 
will  be  stated  here  as  a  theorem  for  convenient  reference: 

THEOREM(2-14) :  If  the  input  to  a  random  T-periodic  system  is  WSS,  then 
the  output  is  T-CS. 

This  theorem  is  a  generalization  of  Theorem  (2-4)  which  applies  to 
deterministic  periodic  systems  and  serves  as  a  basis  for  the  generation 

r 

of  many  CS  processes  of  interest.  This  more  general  theorem  (2-14)  also 
serves  as  a  basis  for  the  generation  of  CS  processes,  and  to  illustrate 
this  we  give  two  examples  of  random  periodic  systems: 

MGDEL(19):  Random  periodic  attenuation  of  a  WSS  process.  The  random 
periodic  attenuator  is  a  useful  model  for  some  communication  channels 
and  for  some  modulators , and  has  a  random  impulse-response  function  with 
realizations  of  the  form 

g(t,x)  =  p(t)fi(t-T)  > 

where  p  is  a  random  periodic  function.  Thus,  the  random  system  function 


takes  the  form 
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H(t,f)  =  /  p(t)6(T)e"^2irfTdx 
—00 

n 

where  (p^)  are  the  random  Fourier  coefficients  for  p. 

Another  useful  model  for  communication  channels  is  a  deterministic 
periodic  attenuation  followed  by  a  random  dispersion.  The  random 
impulse-response  function  for  this  model  has  realizations  of  the  form 

g(t,t)  =  p(T)go(t-i) 

where  p(t)  is  the  periodic  attenuator  and  gQ  is  the  random  time-invariant 
impulse-response  function.  Thus, 

00 

H(t,f)  =  /  p(t-T)g  (T)e'j2ltfTdT 

-00 

=  l  Go(f+n/T)pnej2imt/T 
n 

where  {p^}  are  the  deterministic  Fourier  coefficients  for  p(t)  and  Gq 
is  the  random  transfer  function  for  the  dispersion. 

The  following  is  an  example  of  a  CS  system  which  is  not  a  periodic 
system: 

M0DEL(20j:  Multiplication  of  a  WSS  process  by  a  cyclostationary  process. 

In  communication  systems,  signals  are  sometimes  subjected  to  multiplicative 
noise.  If  the  noise  is  a  CS  process,  then  we  have  a  CS  system  with  impulse 
response  realizations  of  the  form 

g(t,x)  -  n(t)6(t-x) 

where  n  is  a  CS  process,  and  the  system  autocorrelation  function  is 


K  (t,s,f,v)  =  k  (t,s). 
i?  nn 
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c)  Random  time-scale  transformations.  In  Section  2  we  discussed 
deterministic  time-scale  transformations  of  the  form  t'(t)  =  t  +  p(t) 
where  p(t)  was  a  deterministic  periodic  function.  Here,  we  will  consider 
random  time-scale  transformations  with  realizations  of  the  form 
t'(t)  =  t  +  z(t)  where  z  is  a  random  process  which  is  stationary  (or,  in 
some  cases,  cyclostationary)  of  order  two  (S(2)). 

i)  Phase  randomization.  The  simplest  possible  random  time-scale 
transformation  is  the  random  phase-shift:  t'(t)  =  t  +  0,  where  0  is  a 
single  random  variable  with  PDF  p  (•)•  The  phase-shift,  as  defined  here, 

U 

is  identical  to  the  random  delay  which  was  discussed  in  subsection  5a. 
Thus,  we  know  by  Theorem  (2-12)  that  random  phase  shifts  preserve 
cyclostationarity.  That  is,  it  is  not  necessary  that  the  time-origin  or 
phase  of  periodicity  be  known  precisely  (be  deterministic)  in  order  hat 
a  process  be  CS.  In  fact,  the  phase  can  be  a  random  variable  with 
arbitrary  PDF  and  the  process  still  be  CS.  Furthermore,  there  is  an 
interesting  class  of  PDF's  for  the  random  phase  which  reduce  all  CS 
processes  to  stationary  processes.  This  result,  which  is  similar  to  that 
of  Theorem  (2-11)  is  established  in  the  following  theorem; J 8 
THEOREM(2-15) :  For  every  T-CS  random  process  y,  the  random  process  x  with 
realizations  of  the  form  x(t)  =  y(t+o)  is  WSS  if  and  only  if  the  PDF  pQ(*) 
for  the  independent  random-phase  variable,  with  realization  o,  satisfies 
the  condition: 


18This  theorem  (in  a  slightly  different  form)  has  been  stated  and  proved 
by  liurd  [5],  and  the  "sufficient"  portion  has  jeen  proved  and/or  demonstrated 
for  specific  processes  by  a  number  of  authors  [1,2, 3, 7].  In  addition,  Hurd 
has  given  an  interesting  companion  theorem  which  states  that  a  process  is 
CS  if  and  only  if  its  phase-randomi2ed  version  is  WSS. 
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I  P  (o-nT)  =  1/T  V-a; 
n=-°° 

and  if  the  PDF  is  constrained  to  be  duration  limited  to  (-1/2T,1/2T), 
then  this  necessary  and  sufficient  condition  reduces  to 

1/T,  j  o J  <  1/2T 
0  ,  |o|  >  1/2T 

Proof : 

i)  The  mean  for  x  is 

CO 

mx(t)  =  E{y(t+6) }  =  /  m  (t+a)pQ (o)do. 

—  00  « 

Now.  since  ny  is  T-periodic,  we  make  use  of  its  Fourier  series 
representation 

f  v  i2un(t+o)/T  ,  .. 

V0  =  I  l  ane  Pe(o)do 

n 

=  I  anPS(„/T)ei2”nt/T 
n 

where  P.(#)  is  the  Fourier  transform  of  pa(*)*  Clearly,  from  this  last 
y  y 

expression,  will  be  independent  of  t  for  every  sequence  {a^}  if  and 
only  if  P  (n/T)  =  0  for  all  n  /  0. 

ii)  The  autocorrelation  for  x  is 

co 

k  (t.s)  =  E{y(t+0)y(s+0)}  =  /  k  (t+o,s+o)p  (o)do. 

-CO  ** 

New,  since  k^(t,s)  is  jointly  T-periodic  in  t  and  s,  we  make  use  of 
its  Fourier  series  representation  (see  Eq.  (2-13)): 

-•  /  l  kn(t-s)e-l27,n(t+<7^/Tpe(o)do 

-®  n 

l  kn(t-s)P*(n/T)ej27!nt/T 


Pe(o)  = 
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Clearly,  from  this  last  expression,  kxx(t,s)  will  be  a  function  of  only 
(t-s)  for  every  sequence  {kn>  if  and  only  if  PQ (n/T)  =  0  fbr  every 
n  +  0. 

iii)  From  the  Poisson  sum  formula  [2] : 

l  p0 (a-m/T)  =  i  \  P0(n/T)e'j2mo/'1 
m  n 

00 

=  y  V°) =  k  f  P0(Y)dY  =  1/T 

.CD 

where  the  second  line  is  a  valid  equality  if  and  only  if  P0(n/T)  =  0 
for  all  n  ^  0.  Hence,  from  i),  ii),  iii),  x  is  WSS  if  and  only  if 
00 

l  p0(0-nT)  =  1/T  ¥-0 

n=-°° 

and  clearly,  if  pQ(*)  is  constrained  to  be  duration  limited  to 
(-1/2T.1/2T) ,  then  this  condition  can  only  be  satisfied  by  the  uniform  PDF 

i/T,  | a |  <  1/2T 
0  ,  i o |  >  1/2T. 

QED 

This  next  theDrem  establishes  a  rather  obvious,  but  quite  useful, 
result  on  the  relationship  between  the  statistics  of  a  phase-randomized 
process  and  the  time-averaged  (smoothed)  statistics  of  its  deterministic- 
phase  counterpart. 

THEOREM(2-16) :  The  first  and  second  order  statistics  of  the  process  x 
obtained  by  phase-randomizing  a  process  y,  with  a  random-phase  FDF  pQ  ( * ) , 
are  identical  to  the  time-averaged  (smoothed)  statistics  of  y  if  the 
weighting  function  used  for  averaging  is  p  (•)■■- 


Pq (°)  = 


Proof:  For  any  function  f(*)  we  have 


E{f(x(t*0))}  *  /  E{f(x(t+o))}pe(o)do 
-00 

00  00 
-  /  E{ f (x (t+o) ) }p. (o)do/  /  p0(o)do, 

.9  -0* 

and  this  last  expression  is  the  time-averaged  value  of  E{f(x(t))}  with 
weighting  function  pQ(*)»  and  f or  any  function  g(',‘)  we  have 


E{g(x(t+6),x(s+o))}  ■  /  E{g(x(t+o),x(s+o))}p0(o)do 

-00 

Ob  00 

*  /  E{g(x(t*o),x(sto))}p  (o)do/  /pQ (a)do 

-00  „  00 


and  this  last  expression  is  the  time-averaged  value  of  E{g(x(t) ,x(s)) } 
with  weighting  function  pQ(*). 

QEO 

A  direct  result  of  this  theorem  is  the  fact  that  the  mean  and 
autocorrelation  functions  for  the  WSS  process  x  obtained  by  time¬ 
averaging  over  one  period  the  moan  and  autocorrelation  functions  for  any 
T-CS  process  y: 

T/2 

■x  -  i  J  rov(t)dt 

i  _T/2  y 

1  tV2 

k  (t-s)«=  /  k  (t*o.s*o)do, 

are  identical  to  the  mean  and  autocorrelation  functions  for  the  WSS 
process  x’  obtained  by  phase-randomizing  y  with  a  random  phase  variable 
uniformly  distributed  over  one  period  1/ JT,  1/2TJ . 

The  three  preceding  results  generalize  from  one-dimensional  random 
signal  processes  to  two-dimensional  random  system  processes  as  follows: 
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THEOREM(2-17) :  For  every  T-CS  system  G'  with  random  system  function 
H'(t,f),  the  random  system  G  with  random  system  function  realizations  of 
the  form  H(t,f)  *  H'(t*o,f)  where  o  is  the  realization  of  an  independent 
random-phase  variable  with  arbitrary  PDF,  is  also  a  T-CS  system. 
THEOREM(2-18) :  For  every  T-CS  system  G',  the  corresponding  phase- 
randomized  system  G  is  WSS  if  and  only  if  the  PDF  p0(*)  for  the  independent 
random-phase  variable  satisfies  the  condition 

oo 

l  pe(o-nT)  -  1/T  *  0  , 

n«-«> 

and  if  the  PDF  is  constrained  to  be  duration  limited  to  (-1/2T.1/2T) , 
then  this  necessary  and  sufficient  condition  becomes 

1/T,  |o|  <  1/2T 
0  ,  | o |  >  1/2T. 

THEOREM (2- 19) :  The  system  autocorrelation  function  for  the  random  system 
G  obtained  by  phase- randomizing  a  system  G' ,  with  a  random-phase  PDF  pQ(*) 
is  identical  to  the  time-averaged  (smoothed)  system  autocorrelation 
function  for  G',  if  the  weighting  function  used  for  averaging  is  P0(*)* 

The  proofs  of  these  last  two  theorems  directly  parallel  those  for 
Theorems  (2-15),  (2-16),  and  will  not  be  repeated  here.  The  proof  of 
Theorem  (2-17)  is  trivial. 

As  a  result  of  Theorems  (2-11),  (2-16),  (2-19),  we  soe  that  when  it 
is  appropriate  in  specific  analyses  to  ignore  periodic  fluctuations  in 
random  cyclostationury  signals  and  systems  and  in  deterministic  periodic 
signals  and  systems,  then  these  fluctuations  can  be  "averaged  out"  either 
by  uniformly  randomizing,  over  one  period,  the  time-origin  or  phase  of  the 


p0(o)  • 


periodic  fluctuations,  or  by  tine-averaging,  over  one  period,  the 
fluctuations  with  a  uniform  weighting  function. 

Also,  from  the  discussion  at  the  beginning  of  this  subsection  and 
Theorem  (2-17),  we  see  that  it  is  not  necessary  that  the  time-origin  or 
phase  of  periodicity  be  known  exactly  (be  deterministic)  in  order  that  a 
signal  or  system  be  cyclostationary.  In  fact,  the  phase  can  be  a  random 
variable  with  arbitrary  PDF. 

ii)  Generalized  angle  modulation.  We  now  consider  more  general  random 
time-scale  transformations. 

THEOREM (2- 20):  A  random  time-scale  transformation  G,  with  realizations 
of  the  form  t *  (t )  =  t  <■  z(t)  where  z  is  a  S(2)  process  with  joint  PDF  for 
z(t)  and  z(s)  denoted  p  (•,*)•  has  system  autocorrelation  function 

fc*S 

KG(t,s,f,»)  -  P*_s(t» 

where  P„  „  is  the  double  Fourier  transform  of  p_  ,  and  the  transformation 
t-s  rt-s 

G  preserves  cyclostationarity. 

Proof:  The  impulse-response  function  for  G  has  realizations  of  the  form 
g(t,x)  6(T~t-z(t)) 

so  that 

h{t,T)  •  6(t*z(t)) 
and 

H(t,f)  .  ei2,z<t)f  . 

Now,  the  system  autocorrelation  function  is 

K(.(t,«>,f,v)  *  If  e-’2nf°  e"^2nvY  Pt_s(o,Y)dodY 


P!  „(*>). 

t-a 
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This  system  autocorrelation  function  clearly  satisfies  the  hypothesis 
of  Theorem  (2-12);  hence,  the  system  G  preserves  the  cyclostationarity 
of  zero-mean  processes.  But  since  the  mean  of  the  output  x  satisfies 

0» 

mY(t)  «  /  m  (t+o)p(o)do 
•00  * 
to 

•  I  m  (t+T+o)p(o)do 
•  00  * 

*  mx(t+T). 

where  y  is  the  input  to  G  and  p(«)  is  the  PDF  for  z(t)  for  all  t,  then 
G  also  preserves  cyclostationarity  of  non-zero-mean  processes. 

QED 

The  degenerate  case  in  which  the  input  process  is  a  deterministic 
T-periodic  signal  is  actually  generalized  angle  modulation,  and  is  of 
special  interest  in  communications.  For  convenient  reference,  we  restate 
the  above  theorem  for  this  special  case: 

THEOREM (2-21 ) :  If  a  deterministic  T-periodic  signal  q(t)  is  subjected 
to  a  random  time-scale  modulation  with  realizations  of  the  form  t'(t)st  ♦  z(t) 
where  z  is  S(2),  then  the  resultant  process  x  with  realizations  of  the 
form  q(t  ♦  z(t))  is  T-CS. 

Examples: 

M0DEL(21):  Conventional  phase-modulation  (PM).  If  q(t)  is  a  sinusoid, 
then  x  is  a  conventional  phase-modulated  signal  where  z  is  the  modulating 
signal. 

MODEL(22):  Conventional  frequency-modulation  (FM).  If  q(t)  is  a  sinusoid, 
and  if  the  realizations  of  z  take  the  form 
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t 

z(t)  *  /  z0(o)do 

then  x  is  a  conventional  frequency  nodulated  signal  where  zQ  is  the 
modulating  signal. 

Variations  on  these  two  conmonly  used  communication  signal  formats 
are  those  of  digital  PM  and  digital  PM,  the  statistics  of  which  have 
been  partially  analyzed  by  various  authors  [23],  [24].  The  following 
theorem  establishes  the  cyclostationarity  of  digital  angle-modulated 
signals  under  a  stationarity  assumption  which  differs  from  that  in  the 
above  theorem. 

THEOREM(2-22):  If  a  deterministic  T-periodic  signal  q(t)  is  subjected 
to  a  random  time-scale  modulation  with  realizations  of  the  form 
t'(t]  *  t  ♦  z(t)  where  z  is  a  PAM  signal  with  realizations  of  the  form 

z(t)  «  l  a  p  (t-nLT) 
n 

where  [an)  is  a  S(2)  random  sequence  and  L  is  an  integer,  then  the  resultant 
signal  x  with  realizations  of  the  form 

x(t)  «  q(t  ♦  l  anpo(t-nlT)) 
n 

is  CS  with  period  LI. 

Proof: 

00 

i)  mx(t)  «  /  q(t  ♦  l  opo(t-nLT))p(o)do 

n 

00 

=  /  qft  ♦  LT  ♦  o  J  p  (t  ♦  LT  -  (n*l)LT))p(o)dcj 

n 

00 

=  /  q(t  ♦  LT  ♦  o  V  P  ft  ♦  LT  -  n'LT))p(o)do 

n‘  0 

=  rax(t  ♦  LT)  V  t , 
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where  p(*)  is  the  PDF  for  &n  for  all  n, 

ii) 

m 

kxX(t*s)  *  W  *(*  +  l  opo(t-nLT))q(s  +  l  YPfl (s -mLf) )pn  B(o, y)dodY 
-®  n  m 

m 

•  //  q(t  ♦  LT  +  o  l  p  (t  ♦  LT  -  (n+l)LT))* 

-*  n 

q(s  *  y  l  P0(s  ♦  LT  -  (»+l)LT))p(n+1)(n+1)(e,Y)dodY 

m 

-  kxx(t  ♦  LT,s  +  LT)  ¥  t.s. 


where  p  (*,*)  is  the  joint  PDF  for  a  and  a  for  all  n  and  m.  Hence, 
n-n  n  n 

from  i)  and  ii),  x  is  CS  with  period  LT. 

Q'ED 


Examples: 

MODEL (23):  Digital  phase-modulation  (DPM).  If  q(t)  is  a  sinusoid,  then 
x  is  a  conventional  digital  phase-modulated  signal  with  modulating 
sequences  {a^},  and  pulse-shape  pQ(t). 

MODEL (24):  Digital  frequency-modulation  (DFM).  If  q(t)  is  a  sinusoid, 
and  if  the  "pulses"  pQ(t)  take  the  form 

t 

P0(t)  *  /  Po(o)do» 

■V 


then  x  is  a  conventional  digital  frequency-modulated  signal  with 
modulating  sequence  {*n)>  and  pulse-shape  p^(t). 

Finally,  we  point  out  that  random  time-scale  transformations  can 
be  CS  systems  in  that  they  can  transform  WSS  inputs  into  CS  outputs: 
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THEOREM (2-23):  The  output  x  of  a  random  time-scale  transformation  G 
with  realizations  of  the  form  t*(t)  =  t  ♦  z(t),  where  z  is  a  T-CS(2) 
process,  is  T-CS  if  the  input  y  is  WSS. 

Proof:  Hie  system  autocorrelation  function  for  G  is  (as  shown  in  the 
proof  of  Theorem  (2-20)) 

KG(t,s,f,v)  a  PJ  (f.v). 

where  P  is  the  double  Fourier  transform  of  p  which  is  the  joint 
t |S  v  |S 

PDF  for  z(t)  and  z(s).  But  since  z  is  T-CS (2)  then  pt+T  *  pt  s  for 
all  t  and  s,  so  that  satisfies  the  hypothesis  of  Theorem  (2-13),  and 
G  preserves  cyclostationarity  of  zero-mean  inputs.  But  since  the  mean 
for  the  output  x  takes  the  form 

on 

mx(t)  =  /  m  pt(o)do  «  m  V-  t 

.<»  *  * 

=  m  (t  ♦  T) 
x 

where  pt  is  the  PDF  for  z(t),  then  G  also  preserves  the  cyclostationarity 
of  non- zero-mean  inputs  y. 

QED 


.  .  Jx  r/» y-s;. ’’*?  '  ‘“V/’1’  *  *V*" 
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6.  Random  Multi -dimensional  Linear  Transformations  (Random  Scanning) 

In  this  section,  we  briefly  present  a  generalization  of  the  ideas 
presented  in  Section  3  on  deterministic  multi-dimensional  linear 
transformations.  The  motivation  here  is  to  develop  a  theory  upon  which 
studies  of  random  scanners  can  be  based.  We  use  the  same  definitions  and 
input-output  relation  (Eq.  (2-21))  as  in  Section  3;  however,  we  now  allow 
the  three-dimensional  impulse-response  function  to  be  a  (four-dimensional) 
random  process.  Thus,  generalizing  our  definition  of  "quasi -periodic 
transformation"  (Section  3,  Eq.  (2-22))  and  our  definition  of 
"cyclostationarv  system"  (Section  Sb),  we  define  the  transformation  G  to 
be  quasi-cyclostationary  with  period  T  if  and  only  if  the  generalized 
system  autocorrelation  function  satisfies  the  cyclostationarity  condition: 

KG(t+T,s+T,f,f,u1,v1,u2,v2)  »  K(.(t,s,f,f,u1,v1,u2,v2) 

for  all  t.s.f.Uj.Vj.u^.Vj  where  is  defined  as  in  Eq.  (2-30)  of 
Section  S.  (In  most  applications  Uj,  u^,  vJt  v2  will  be  position 
coordinates;  t,s  will  be  time  variables,  and  f  will  be  the  frequency 
variable.) 

Now,  we  have  the  following  generalization  of  Theorems  (2-6), (2-13): 

THEOREM (2- 24 ) :  If  the  input  y  to  a  three-dimensional  quasi-cyclostationary 
transformation  with  period  T  is  a  zero-mean  quasi -wide-sense-stationary 
process,  then  the  one-dimensiomU  output  process  x  is  T-CS. 

The  proof  of  this  theorem  is  a  simple  extension  of  that  for  Theorem  (2-13). 
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For  applications  of  this  theorem,  we  have  the  generalizations  of 
those  applications  of  Theorem  (2-6)  in  Section  3:  i.e. ,  random 
scanners.  Consider,  for  example,  the  line  scanner  of  Section  3,  but 
with  the  random  window  function 

g(t,T,u,v)  -  gCUjUJ-u^jCO-vWt-t), 

where  g  is  a  two-dimensional  random  function,  and  and  u^  are 
one-dimensional  random  processes.  If  g  is  statistically  independent  of 
Uj  and  Vj  which  are  either  jointly  CS(4)  (cyclostationary  of  order  4) 
or  are  independent  and  CS(2),  then  the  random  line-scanner  G  satisfies 
the  hypotheses  of  Theorem  (2-24).  Hence,  random  line-scanners  with 
random  (randomly  selected)  window  shapes  and  random  CS  window  velocities 
generate  CS  signals  from  WSS  inputs. 

Thus,  we  see  that  the  precise  (deterministic)  periodicity  used  in 
the  video  line-scanning  model  of  Section  3  is  not  necessary  in  order  for 
the  resultant  video  signal  to  be  modeled  as  CS. 

More  generally,  we  see  that  signals  obtained  from  various  scanning 
operations  can  be  modeled  as  cyclostationary  even  if  there  is  no  precise 
(deterministic)  periodicity  in  the  model  of  the  scanner. 


\ 


f . 

•  I  ''  ' 

7.  Nonlinear  Random  Transformations  (Jitter) 

\  , 

Unlike  the  random  linear  transformations  of  the  preceding  sections, 

i 

there  does  not  exist  one  simple  characterization,  such  as  a  system  function, 
fbr  the  random  nonlinear  transformations  of  interest  in  this  thesis.  The 

i 

periodic  Volterra  series  representation  introduced  in  Section  4  on 

»  / 

deterministic  nonlinear  transformations  could  be  generalized  here  by 
allowing  the  kernel  functibns  to  be  random;  however,  such  a  representation 
would  not  serve  the  purposes  of  this  thesis. 

In  this  section, ,thep,  we  briefly  extend  the  results  on  deterministic 
izero-raemory  nonlinearities  presented  in  Section  4,  and  then  turn  to  the 
important  issue  of  jitter.  We  characterize  this  random  timing  disturbance 
in  general  terms,  and  then  shov  that  it  preserves  cyclostatioparity  under 

!  •  i 

a  fairly  liberal  set  of  conditions. 

a)  Zero-memory  random  nonlinearities.  A  nonlinear  random  transformation 

.  i  ‘  1 

is  said  to  have  zero-memory  if  and  only  if  for  every  member  of,  its 
ensemble  of  transformations,  every  image  function  evaluated  at  ai>y  time, 

j 

say  t  ,  depends  on  the  corresponding  inverse  image  function  evaluated 

i°  • 

only  at  the  same  time,to.  Realizations  of  the  input  and  output  of  such 
transformations  are  related  as  follows 

I 

I  . 

x(f )  =  G(y(t),,t)  1  . 

where  G ( • , • )  is  a  realization  of  the  random  transformation  G.  1 
We  begin  with  a  theorem  which  establishes  a  class  of  such 
transformations  which  generate  CS  processes  from‘S(2j  processes: 


i 
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TKEOREM(2-25):  If  the  input  y  to  a  zero-memory  randoa  nonlinearity  G 
15  S(2),  and  if  G  satisfies  the  cyclostationarity  condition: 

E{G(o,t)}  «  E{G(o,t+T)}  Yt,a 

E{G(o,t)G(Y,s)}  -  E{G(o,t+T)G(Y,s+T)}  V-  t,s,o,y, 

then  the  output  x  is  T-CS. 

Proof: 

»x(t)  •  E{G(y(t).t)} 

to 

*  /  E{G(o,t)}p(a)do 

•on 

m 

s  /  E{G(o,t<T))p(o)do 
.00 

*  »x(t+T), 

where  p(*)  is  the  PDF  for  y(t)  for  all  t,  and 
fcxx(t,s)  *  E{G(y(t),t)G(y(s),s)} 

CO 

a  //  E{G(o,t)G(y,s) jp  (o,Y)dody 

•oo  v-S 

00 

°  JJ  E{G(o’t+T)GfY’SfT))P(t+T)-(s+T)^»Y)dodY 
3  kxx(t+T*s+T)  *  t.s, 

where  Pt_s(*»*)  is  the  joint  PDF  for  y(t)  and  y(s).  Hence  x  is  T-CS. 

QED 

As  an  example  of  a  cyclostationary  zero-memory  nonlinearity,  consider 
the  very  elementary  transformation  of  "additive  noise": 

(y{t),t)  -  y(t)  ♦  n(t),  where  n  is  a  CS  random  "noise"  process.  G  clearly 
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satisfies  the  cyclostaticnarity  condition  of  the  theorem.  In  fact,  the 
sun  of  any  nuaber  of  statistically  independent  MSS  and  T-CS  processes 
will  be  T-CS. 

b)  Jitter.  The  various  repetitive  operations  such  as  sampling, 
scanning,  and  auitiplexing,  to  which  signals  are  often  subjected,  are— 
of  course— never  exactly  periodic.  In  sone  analyses,'  this  departure  froa 
exact  periodicity  can  be  neglected;  however,  in  the  event  that  it  is— 
or  nay  be — significant,  this  departure  froa  exact  periodicity  should  be 
aodeled  so  that  it  can  be  dealt  with  analytically. 

We  have  seen,  throughout  this  chapter,  a  nuaber  of  aodels  for  T-CS 
processes  with  realizations  of  the  fora 
« 

y(t)  *  I  x  (t-nT) 
n»-« 

where,  for  exaaple,  the  translates  {xn(t-nT)J  might  be  one-line  segments 
from  a  line-scanned  rectangular  field  as  in  Video,  or  finite  length 
records  from  a  multitude  of  sources  as  in  time-division  multiplexing 
schemes,  or  individual  random  pulses  as  in  the  various  synchronous  pulse 
modulation  schemes  such  as  PAM,  FSK,  and  others,  For  CS  processes  with 
realizations  of  this  form,  we  model  departures  from  exact  periodicity  by 
incorporating  a  random  epoch-jitter  variable  $n  in  each  translate  so  that 

x(t)  *  l  xn(t-nT+fin) 
n 

where  {6^}  is  a  random  sequence.  Hence  x  is  obtained  from  y  via  a  random 
nonlinear  transformation  with  memory. 

Now,  two  fundamental  models  for  this  sequence  { 6^ }  are  of  importance, 
in  that  each  accurately  represents  one  of  the  two  most  common  physical 
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mechanisms  which  cause  jitter: 

(1)  each  translate  is  jittered  w.r.t.  the  time-origin  of  the  total 
process  (  t  «  0  ), 

(2)  each  translate,  say  the  n^,  is  jittered  w.r.t.  the  time-origin  of 

n-1 

the  preceding  translate  (  t  ■  (n-l)T  +  T  6.) . 

i-0  1 

In  the  first  case,  where  "jitter"  is  the  more  appropriate  terminology, 
the  random  sequence  {6R}  can  be  modeled  as  a  stationary  finite-variance 
sequence  with  autocorrelation  function  denoted  k^(n-m) .  Then  the  second 
case,  for  which  the  term  jitter  is  somewhat  inappropriate,  can  be  modeled 
by  a  new  sequence  { vn )  which  is  composed  of  the  partial  sums  of  the  (6^): 


Thus,  {y^}  is  a  discrete  Wiener  process  (Wiener  sequence)  and  is  non* 
stationary  with  growing  variance.  Due  to  the  difficulties  associated  with 


i 
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this  model,  we  will  simply  state  that  the  corresponding  jittered  process 
x  is  not,  in  general,  CS;  but,  in  the  event  that  the  process  is  observed 
by  a  tracking  receiver,  then  the  observed  process  can  be  remodeled-- 
using  the  time-varying  frame  of  reference  provided  by  the  tracker— in 
terms  of  the  first  model  for  jitter  (mentioned  above)  which  is  more 
tractable. 

The  new  text  book  Theory  of  Synchronous  Communications  by  J.J.Stifler 
[40]  provides  a  comprenensive  tutorial  treatment  of  synchronization:  the 
problems  of  detecting  and  estimating  symbol  epochs  and  carrier  phase,  and 
of  tracking  fluctuations  in  periodicity,  and  related  synchronization 
problems  are  treated  using  the  theories  of  maximum-liklihood  detection  and 
estimation,  and  familiar  communications  components  such  as  matched  filters 
or  correlators  and  phase- lock  loops. 

Returning  now  to  the  first  model  for  jitter,  we  establish  the 
preservation  of  cyclostationarity  with  the  following  theorem: 

THEOREM (2- 26 ) :  If  a  T-CS  process  y  with  realizations  of  the  form 

o» 

y(t)  «  l  x  (t-nT) 
n«-~ 

and  autocorrelation  function  of  the  form 

kw(t,s)  *  l  k  (t-nT.s-mT) 

^  n,m  n*m 

is  modified  to  include  jitter: 

x(t)  »  l  xn(t-nT*«nJ 
n 

where  the  random  jitter  sequence  is  S (2),  then  the  jittered  process 
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Proof: 


\(t> 


'  l  J  Elx«(‘-nT*»11)>P&>nM«n 

m 

*  f  E<1  x  (t-nT+o)}p(o)do 
-•  n 

m 

'  f  E(y(t*o))p(o)do 


■  f  ®v(t+°)p(o)do 
•«  + 

m  i 

1  /  »v(t+T+o)p(o)do 

mm  * 


«x(t+T) 


where  p(‘)  is  the  PDF  for  «n  for  every  n. 
ii) 


kxx(t'S)  “  nl,JJ  EtX«(t-"T*,,n)x.<s-Tn.))pn.,(on.y,)<landyi 

0» 

*  E{  nf*  Xn(t’nT+o)x«Cs’BT+Y)}pn.B(o,Y)dodY 

m 

‘  JJ  J.  ‘„..(t-»T*«,s-«Tn)PI1.1>(o.Y)dadY 

00 

*  JJ  n[B  k(n*l)-(«+i)(t+T‘(n+1iT+0»s+T-(m*l)Tn). 

P(n+l)-(n*l)(o'Y)dodY 

00 

*  JJn'lm'  kn,m,(tfT’n'Tfo»s*T‘n,T+Y)pn,_B,(o,Y)dodY 


3  ^(t+T.s+T) 


*  t,s, 


“here  is  th£  J°‘«  pdf  for  «n  and  Jn  for  all  „  and  «.  Non, 

from  i)  and  ii),  x  is  T-CS. 


QED 


As  examples,  we  merely  note  that  the  hypotheses  of  this  theorem 
are  satisfied  by  the  time-division-multiplexing  scheme  (TDMj)  with 
independent  sources,  by  the  video  model  derived  by  Franks  (see  Sec.  3), 
and  by  all  the  synchronous  pulse-modulation  schemes,  with  S(2)  modulating 
sequences,  such  as  PAM,  FSK,  PKM,  and  others  (see  Sec.  4). 

We  now  give  an  extension  of  this  theorem  to  CS  random  systems: 

THEOREM (2-2 7):  If  a  T-CS  random  system  G',with  system  function  realizations 
of  the  form 

m 

H' (t,f)  ■  l  H  (t-nT,f) 
n«-“ 

and  system  autocorrelation  function  of  the  form 

Kg,(t,s,f,v)  *  l  JC  (t-nT,s-mT,f,v* . . 
n,m 

is  modified  to  include  jitter 

H(t.f)  •  I  Hn(t-nT+«n,f) 
n 

where  (6^)  is  S(2),  then  the  resultant  jittered  system  G  is  T-CS. 

The  proof  directly  parallels  part  ii)  for  Theorem  (2-26)  and  will 
not  be  repeated  here.  Note  that,  for  the  special  case  of  deterministic 
signals  and  systems,  the  two  preceding  theorems  establish  the  fact  that 
T-periodic  signals  and  systems,  when  jittered,  become  T-CS  signals 
and  systems. 

In  conclusion,  we  emphasize  the  fact  that  there  need  be  no  exact 
periodicity  in  the  realizations  of  a  CS  process:  the  occurrence  of  pulses 
in  jittered  PAM,  for  example,  is  not  periodic,  nor  are  the  pulse 
occurrences  in  a  Poisson  pulse  process  with  periodic  rate  parameter. 


The  periodicity  neod  only  be  present  in  the  first  and  second  order 
ensemble  averages.  Yet,  as  pointed  out  a  number  of  tines  in  both  this 
chapter  and  Chapter  I,  the  nean  and  variance  for  a  CS  process  can  be 
constant  so  that  the  periodicity  in  these  quantities  is  degenerate,  and 
still  such  processes  can  exhibit  strong  periodic  fluctuations  in 
correlation,  as  exemplified  by  the  tine-division-multiplex  (TDM^  or  TDMj) 
of  WSS  processes  with  equal  neans  and  variances  but  with  disjoint 
PSD's,  or  as  illustrated  in  Figure  (1-3)  for  PAN. 


Figure (2-1)  Diagram  of  sy  stem  (t rans format i on )  G  with  input 
(inverse-image)  y  and  output (image)  x. 
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Figure (2-2)  Structural  diagram  of  the  Fourier  series  representation 
of  a  periodical ly(T)  tine- varying  linear  system. 
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Figure (2-7)  a)  Utaltered  process,  b)  Transformed  process 
obtained  by  compressing  T-length  records  into  T/ft- length 


records. 
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Figure (2- 10)  Circle-scanning  diup.ran(r  is  an  "effective" 
reception  radius.) 


CHAPTER  III 


SERIES  REPRESENTATIONS  FOR  CYCLOSTATIONARY  PROCESSES 
(AND  THEIR  AUTOCORRELATION  FUNCTIONS) 

1 .  Introduction 

Many  analyses  involving  random  processes  are  greatly  facilitated 
by  the  use  of  appropriate  representations  for  the  processes  and  their 
autocorrelations  functions.  These  representations  are  frequently 
categorized  into  one  of  two  classes:  "integral"  representations  and 
"series"  representations.  It  is  the  latter  class  which  concerns  us  here. 
In  this  chapter  we  develop  two  types  of  series  representations  for 
cyclostationary  processes: 

(1)  Discrete  "translation  series"  representations  of  the  form 
m  M 

x(t)  «  l  l  a  ♦  (t-nT)  V-tc  (-,-), 

n=-“  p*l  v  * 

where  )  are  deterministic  basis  functions,  and  ((a  };  p»l,2,..,M) 

P  nP 

are  jointly  wide-sensc-stationary  discrete  random  processes  (random 
sequences)  referred  to  as  "representors". 

1.2)  Continuous  series  representations  of  the  form 


M 

x(t)  *  l  a  (t)l  (t)  y  t  c  (-»,•>), 

p«l  p  p 


jointly  wide-sense-stationary  (or  cyclostationary)  continuous-time  random 
processes  referred  to  as  the  representors. 
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One  of  the  advantages  typically  gained  in  representing  one  process 
in  terns  of  a  multiplicity  of  others  is  that  these  other  processes 
(representors)  are  chosen  for  convenience  of  analysis  and/or  synthesis. 

For  example,  the  translation  series  representation  employs  representors 
which  are  not  only  discrete,  but  also  jointly  wide-sense-stationary. 
Similarly,  the  continuous  series  representations  developed  in  this  chapter 
employ  simple  continuous-time  representors  such  as  piecewise  constant 
processes  with  constant  means  and  variances,  or  jointly  wide-sense- 
stationary  bandlimited  processes.  As  shown  in  the  next  chapter,  such 
representations  enable  us  to  convert  single-variable  problems  with 
periodically  varying  parameters  to  multi-variable  problems  with  constant 
parameters. 

All  representations  presented  in  this  chapter  are  valid  on  the 
infinite  interval  (-*,•),  and  since  all  WSS  processes  are  CS,  then  the 
representations  apply  to  all  WSS,  as  well  as  all  CS,  processes. 

Before  delving  into  the  topic  of  representation,  we  first  define  a 
Hilbert  space  of  random  processes  that  will  provide  a  framework  within 
which  we  can  unambiguously  present  our  results  on  representation.  We  begin 
with  thr  definitions  of  two  well  known  Hilbert  spaces: 

(1)  Hilbert  space  of  random  variables:  We  define  HRV  to  be  the  Hilbert 
space  of  finite  mean-square  random  variables  with  the  following  inner 
product 

(x.y)tL  -  E{xy*} 

kV 

and  induced  norm 

IMIu  -  tE(lx|2jj‘/2. 

nRV 


(2)  Hilbert  space  of  functions:  We  define  L2[a,b]  to  be  the  Hilbert 
space  of  sqiare-integrable  functions  on  the  interval  [a,b]  with  the 
following  inner  product 

a  b 

(*»y)L2  -  /  *(t)y*(t)dt 
and  induced  norm 

I|x|Il2  5  l  f  |x(t) 1 2  dt]1/2. 
a 

We  remind  the  reader  that  the  elements  in  infinite-dimensional 
Hilbert  spaces  such  as  these  are  equivalence  classes,  each  of  which  may 
be  comprised  of  an  uncountable  number  of  equivalent  elements,  where  two 
elements  are  equivalent  if  and  only  if  they  are  equal  modulo  the  set  of 
zero-norm  elements  [2].  For  example,  the  uncountable  set 
;  x(t),  t  /  c.,  t  e  {a,bj 

x.(t)  »  !  V-c,  e  [a,b] 

X  0,  t  •  Cj 

is  an  equivalence  class  (  and  therefore  a  single  element)  in  L2[a,b] 
since  the  norm  of  the  difference  between  any  two  members  is  zero.  The 
terminology 

Xi(t)  3  Xj(t)  a.e.  t  e  (a.bj 

which  is  read  "x^  equals  x^  for  almost  every  t  contained  in  the  interval 
[a,o]"  is  often  used  to  denote  equivalence  in  L2[a,b],  and  will  be  used 
here  occasionally;  however,  for  the  most  part,  the  a.e.  will  be  suppressed. 
In  the  terminology  "mean-square  equivalent"  is  frequently  used  to 
denote  the  nature  of  equality  in  that  space,  and  will  be  used  here 
occasionally. 


Now,  although  periodic  functions  are  not  contained  in  the  Hilbert 
space  L2  (-«•,») ,  they  can  be  made  into  a  related  Hilbert  space  of  their  own: 

(3)  Hilbert  space  of  periodic  functions:  We  define  L2^ (“*»")  t0  be 
the  Hilbert  space  of  square- integrable  (on  [0,T])  periodic  functions  on 
(„«,«•)  with  the  commensurable  periods  (Tn;  Tn  «  T/n,  n  «  1,2,...} 

with  the  following  inner  product 

A  T 

(x,y).2-  /  x(t)y*(t)dt 
LP  0 

and  induced  norm 

I  Win  *  I /VolW'2. 

T»  0 

The  cyclic  character  of  the  functions  in  this  space  is  directly  responsible 
for  the  existence  of  an  L2-type  of  inner  product  and  norm  which  are  valid 
for  functions  on  (-»,«•)  which  are  not  in  L2 (-»,«) .  Although  this  Hilbert 
space  is  a  trivial  variation  on  (and,  in  fact,  isomorphic  to)  L2[0,T],  it 
leads  directly  to  the  following- -far  more  interesting- -Hilbert  space  of 
cyclostationary  processes: 

(4)  Hilbert  space  of  cyclostationary  processes:  We  define  ^(T) 

to  be  the  Hilbert  space  of  all  finite  mean-square  jointly  CS  processes  on 

(-•,»)  with  the  commensurable  periods  fTn ;  Tn  *  T/n,  n  *  1,2, . } 

with  the  following  inner  product 

a  T 

(x,y)H  -  /  E{x(t)y*(t)}dt 
nCS  0 

and  induced  norm 

l|x|L  -[  /  E{|x(t)|2)dt)1/2. 

"cs  0 
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Note  that,  paralleling  the  case  for  ,y  the  existence  of  an 
L2-type  of  norm  for  is  a  direct  result  of  the  cyclic  character 

of  the  second  order  moments  of  the  elements  in  our  space;  i.e. , 

E{x(t+T)y*(t+T)}  «  E{x(t)y*(x)}  a.e.  t,t  e  (-•,•) 

for  every  x,y  in  1^,^.  When  visualizing  the  elements  of  this  space, 
recall  that  statistically  independent  and/or  uncorrelated  CS  processes 
are  jointly  CS,  and  that  all  WSS  processes  are  CS  with  arbitrary  period 
T.  In  fact,  the  elements  of  Hcs^  might  be  more  amenable  to  visualization 
if  thought  of  as  functions  of  two  variables  x(t,u)  where  the  section 
function  x(t, *)  is  a  random  variable  with  domain:  the  set  of  events 
(sample  space)  on  which  the  underlying  probability  space  is  defined,  and 
with  range:  the  complex  numbers,  and  where  the  section  function  x (*,w) 
is  a  function  in  the  usual  sense  with  domain:  the  real  numbers,  and 
range:  the  complex  numbers. 

In  concluding  this  introductory  section,  we  emphasize  that  throughout 
this  chapter,  expressions  of  the  form 


x(t)  =  x'(t) 


V  t  e 


--where,  for  example,  x'  is  a  representation  for  the  CS  process  x-- 
should  be  interpreted  as  meaning  that  x  and  x*  are  in  the  same  equivalence 
class  in  Hcg:  i.e.,  the  norm  of  their  difference  is  zero. 


2.  Translation  Series  Representatidns 

Due  to  the  length  of  this  section*,  and  the  fact  that  it  is  the 
heart  of  the- theoretical  content  of  this  dissertation,  we  begin  by  giving 
a  brief  outline  of  the  eight  subsections  included  herein: 

Subsection:  a)  Definition  of,  and  existence  and  identification  theorem 
for,  translation  series  representations. 

b)  Interpretation  of  translation  series  representations  as 
representations  in  terns  of  PAM  processes. 

c)  Implementation  of  the  process -resolution  and 
reconstruction  operations. 

d)  Examples  of  translation  series  representations. 

e)  Classes  of  processes  which  admit  finite  order  translation 
series  representations. 

f)  Application  of  translation  series  representations  to 
the  solution  of  linear  integral  equations. 

g)  Application  of  translation  series  representations  to  the 
realization  of  periodically  time-varying  linear  systems. 

h)  Application  of  translation  series  representations  to  the 
definition  of  a  generalized  Fourier  transform  for 

«  cyclostationary  (and  WSS)  processes. 
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a)  Definition.  A  particularly  interesting  class  of  discrete 

representations  for  cyclostatiohary  processes  is  the  class  of 

translation  series  representations.19  A  cyclostationary  process  x  with 

period  T  is  said  to  admit  a  translation  series  representation  of  order 

M  if  there  exists  a  set  of  M  deterministic  basis  functions 

ftp!  P  *  1,2,...,M},  and  a  set  of  M  jointly  wide-sense-stationary  random 

sequences  {{a  };  p  «  such  that 

np 

m  M 

E(|x(t)  -  l  l  ann+„(t-nT)|2}  *  0  a.e.  t  c  (-%•).  (3-1) 

n=-«  p*l  np  p 

where,  by  jointly  WSS,  we  mean  that  E{anp},  Efa(n+a)paJ|q}  are  independent 
of  n  for  all  p,q.  Thus  a  translation  series  representation  (TSR)  of 
order  M  is  a  representation  of  a  scalar  cyclostationary  continuous-time 
process  in  terms  of  an  M-vector  of  jointly  wide- sense-stationary  discrete¬ 
time  processes. 

The  autocorrelation  function  for  a  process  with  a  TSR  of  order  M 
has  the  corresponding  representation: 


k«(t,s)  ■  L  „ 

p,q*i 


a.e.t,s  e  (-“>,«) 


(3-2) 


where  the  elements  of  the  M  x  M  matrix  (A  )  of  correlation  sequences 

n-m 


are  given  by 


nm  S  ^annaM^’ 

n-m  np  mq 


(3-3) 


•0 
!  0 
'  LI 


19  Representations  of  this  type  have  been  alluded  to  by  Jordan  (25]  and 
Breisford  (19]  and  perhaps  some  others;  but  have  not  (to  ray  knowledge) 
received  more  than  a  brief  mention. 


! 
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and .the  double  Fourier  transform  of  the  autocorrelation  function  has 
the  representation: 

M 


K  r(f,v)  =;  f  l  A  (V)#  (f)4*(v)  l  5  ’f-v+n/T), 
xx  1  p,q«l  P  ^  n*— 

’  I  ! 

where  the  elements  of  the  matrix  A(f)  are  given  by 

i  1  '  *  * 

A  (f)  -  I  fiP*  e-j2nrTf 

pq  --  -  r  . 


(3-4) 


(3-5) 


r*-« 


and  are  z  transforms  of  the  elements  (correlation  seqi  e  ices)  of  (Ar) 

evaluated  at  z  »  e”^2n^.  and  where, 4  is  the  Fourier  transform  of  4  • 

1  P  P 

From  Eq.  (3-3),  we  see  that  the  matrix  of  correlation  sequences 
exhibits  the  symmetry 


(A  )•  «  (A  )* 
r  ;  -  r 


so  that,  from  Eq.  (3-4),  its  z-transform  is  liermitian  symmetric: 


A* (f)  *  A* (f) 


(3-6) 


Using  Theorem  (2<-16)  of  Chapter  II  (on  the  equivalence  between  time¬ 
averaging  and  phase  randomizing),  and  Eq.  (3-2),  it  is  easily  shown  that 

■  1  i  , 

i 

the  power  spectral  density  for  the  stationarized  version  of  a  CS  process 
with  the  above  TSR  has  the  following  quadratic  representation: 


K  (f)  ■  ~  I  A  (f)4  (f)4*(f) 

vv  '  '  T  ^  nn  N  n  7  n'  7 


M 


xx 


T  p,q°i  p<* 


P , 


(3-7) 


The  type  of  separability  exhibited  by  the  TSR  for  the  autocorrelation 

i  t 

function  is  useful  in  solving  integral  equat ions- -as  shown  in  Section  2f 

i 

of  this  chaptcr--which  arise  hi  estimation  and  detection  problems  (where 
the  autocorrelation  function  is  the  kernel  of  a  linear  integral 


I 


transformation),  and  Mill  be  exploited  in  the  following  chapter  on  . 
least-nean-square  estination. 

Most  CS  processes  of  theoretical  and  practical  interest  (those 
contained  in  HcS(t) (-•»•))  admit  translation  series  representations  of 
infinite  order.  Hut  is,  the  norm  of  the  difference  between  such  a 
process  and  an  M**  order  approximate  representation  of  it  monotonically 
approaches  zero  as  M  approaches  infinity.  Hence,  most  CS  processes  of 
interest  can  be  arbitrarily  closely  approximated,  in  norm,  by  a  finite 
order  TSR. 

The  most  generally  applicable  type  of  TSR  is  that  which  employs  a 
complete  orthonormal  (CON)  set  of  basis  functions  <y  in  the  Hilbert 
space  L2[0,T].  Orthonormality  insures  that,  by  definition. 


(VVo ? *Vr|! 

and  completeness  insures  that  Bessel's  equality  [2]  holds: 

-  ¥■  f  e  L2[0,T] 

f(t)  »  l  C'.0L2Mt)  : 

p»l  p  L  P  a.e.  t  c  [0,T] 


(3-8) 


(3-9) 


Note  that  if  we  extend  the  {y  from  L2[0,T)  to  L2(-»,»)  by  defining 


♦p(t)  -  0 


V-  t  t  [Q,T] , 


then  the  set  of  translates  (yt-nT);  p  *  1,2,...;  n  «  0,±1,±2,...} 
is  an  orthonormal  set  in  L2 (-*,»)  in  the  sense  that 


/  yt-nT)^(t-T)dt  .  6pqSm. 


(3-10) 


For  this  class  of  TSH's  we  have  the  following  fundamental  theorem 


on  existence  and  identification: 
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THEOREM(3-l):20Ev«ry  random  process  x  e  (-•,•)  (every  finite 

mean-square  T-CS  process)  admits  the  mean-square  equivalent  TSR: 

m»  m 

x(t)  «  l  l  anDMt“nT)  a-€*  *  c  (-•»*)  (3-11) 

n«-»  p»l  p  ? 

where  the. elements  of  the  jointly  NSS  sequences  (a^)  are: 

T 

an_  *  /  x(t+nT)4*(t)dt  (3-12) 

"P  o  p 

and  where  is  any  CON  set  in  L2[0,T] . 

Proof:  Define 

o(t)  -  E{Jx(t)  -  l  l  8  ♦  (t-nT) |2}. 
n=-»  p*l  np  15 

Expanding  the  square  and  interchanging  summation  and  expectation  yields: 

e(t)  =  k  (t.t)  -  2Re{  £  \  E{an  x*(t))*n(t-nT)J 
**  „p  "P  P 

♦  y  y  n{a  a*  )♦  (t-nT)^(t-mT). 
ntmpTq  npra<»  P  * 

Substituting  the  given  expression  for  a  and  interchanging  integration 
and  'oectation  yields : 

2 OKote  that  any  generally  nonstationary  finite  mean-square  random  process 
can  be  represented  on  (-•»,«)  with  a  TSR.  The  discrete  representors  will 
not,  in  general,  be  jointly  WSS,  but  the  representation  will  be  a  genuine 
discrete  series  representation  in  terms  of  a  countable  number  of 
deterministic  basis  functions  with  random-variable  weights.  However,  in 
contrast  to  the  usual  discrete  representations  on  a  finite  interval,  the 
TSR  on  (-*,«*)  cannot  be  truncated  at  a  finite  number  of  terms  to  provide 
an  arbitrarily  close  (in  mean-square)  approximation  (because  the  sum  over 
the  translation  index  must  always  be  from  to  «°  if  the  representation 
is  to  apply  over  the  entire  interval  (-®,»). 


124 


T 

eft)  «  k  (t,t)  -  2Kef  Ilf  k  (o+nT,t)4*(o)do*  (t-nT)J 
**  npO  *  p  p 

T 

♦  l  l  ff  k  (c+nT, t+»T)4* (a)*  (T)dodT*  (t-nT)$* (t-mT) 
n,«p,qO  p  H  p  H 

or,  changing  the  variables  of  integration: 

(n+l)T 

eft)  «  k  (t,t)  -  2Re[  H  f  k  (o,t)4*(o-nT)do*  (t-nT)J 
xx  n  p  nT  p  p 

fn*l)T  (ra«-l)T 

+  l  l  J  /  k  (o,T)4*(o-nT)4  (T-mT)dodf 
n,m  p,q  nT  »T  p  4 

*  (t-nT)4*(t-mT).  (3-13) 

P  <1 


Now,  since  x  is  a  finite  mean-square  process,  then  the  section  function 


kxx(4,t)  is  in  L2(0,T]  for  every  t  e  (-*,"*),  and  since  x  is  T-CS,  then 
k  is  jointly  T-periodic  in  its  two  arguments  so  that  k  (• • t)  is  in 


xx 


L2(nT,(n+l)Tl  for  all  integers  n  and  every  t  c  (-•*,•).  Also,  by  hypothesis 
(4p(t-nT)}  are  CON  in  L2{nT, (n*l)T) .  Hence,  from  Bessel's  equality 
(Eq.  (3-9)),  we  have: 


™  (n*l)T 


l  l 

p«l  nT 


kxx(o.  t ) ♦ * (o-nT)do4p (t ' -nT)  ■  kxx(t',t) 


for  all  n,  for  a.e.  t'  e  lnT,(n*l)Tj,  and  for  a.e.  t  e  (-•,“),  and  the 
left  member  is  identically  zero  for  t'  t  [nT,(n+l)T].  Hence, 


I  l  I 

n=-«  p=l  nT 


M  fn*l)T 


kxx(a.t)*;(o-nT)do*p(t'-nT)  =  kxx(t',t)  (3-14) 


for  a.e.  t,t'  c  (-<»,“). 

* 

Similarly, 

(m*l)T  Cn*l)T 

l  l  I  III  I  k  (o.i)«M«-nT)At  (t'-nT))- 
m  q  mT  n  p  nT  *  v 

$q(T-mT)dt4*(t-mT) 

fm*l)T 

=  H  J  kxxtt',T)4  fT-nT)d^*(t-mT) 

m  q  ml  '  H  M 


s 
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*  k^t’.t)  (3-15) 

for  a.e.  t,t'  e  (-•»,“). 

Now,  substituting  Eqs.  (3-14,15)  into  Eq.  (3-13)  yields: 

e(t)  «  kjQjft.t)  -  Sk^Ct.t)  ♦  kxx(t,t)  •  0  a.e.  t  e  (-•*>). 

Hence,  x  and  its  TSR  are  mean-square  equivalent  for  a.e.  t  c  (-»,») . 

The  fact  that  the  sequences  (a^)  are  jointly  NSS  can  be  trivially 
demonstrated  using  their  defining  equation  (Eq.  (3-12))  and  the  fact 
that  x  is  T-CS. 

QED 

Note  that  if  we  substitute  Eq.  (3-12)  of  this  theorem  into  Eq.  (3-3), 
then  we  obtain  the  following  direct  formulas  for  the  correlation  matrices 
of  the  jointly  WSS  sequences: 

T 

A**  *  //  k  * (t+rT , x)4* (t ) ♦  (x)dtdt  (3-16) 

r  0  **  PS 

and,  using  Parseval's  relation  (2): 

m 

»  //  Xxx(f.v)4J(f)4q(v)ej2ffrTfdfdv  .  (3-17) 

b)  Representation  in  terms  of  PAM  processes.  An  interesting 
interpretation  of  the  TSR  of  Theorem  (3-1)  results  from  the  following 
manipulation: 

Define  the  periodic  extension  of  ^  as 

0» 

?_(t)  -  l  *0(t-nT) 
p  n*-®  1 


(3-18) 
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and  define  a  gate  function  w  as 


w(t)  - 


1. 

0. 


t  c  [0,T] 
t  t  [0,T]  ‘ 


Now,  we  have 


(3-19) 


♦p(t)  *  )p(t)w(t), 

and  we  can  write  the  discrete  M**1  order  TSR  for  x  as  an  M-tem  continuous 
series  representation  as  follows: 

*(t)  *  I  K  Vp(mt) 

n«-«*  p*l  r  * 

*  i  i  v(t-nT)Vt} 

p  n  r  * 

M  , 

“  l  (3-20) 

p»l  F  v 

where  (ap(t)}  are  pulse-amplitude  modulated  (PAM)  processes  with  full 
duty-cycle  rectangular  pulses: 


m 

ao(t)  *  2  anDW(t‘nT)- 
•  * 


(3-21) 


A  typical  realization  of  one  of  these  PAM  processes  is  shown  in 
Fig.  (1-2)  of  Chapter  I. 

The  autocorrelation  function  for  x  has  the  corresponding  representation 

kxx(t,s)  *  !  A,  (tis)J  (t)fc(s),  (3-22) 

xx  P,q»l  pq  PH 

where 


Apq«‘s>  3  E(ap(t)a*(s)} 

=  l  APqm  w(t-nT)w(s-»T), 


(3-23) 
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and  since  the  correlation  matrix  A(t,s)  is  jointly  T-periodic  in  its 
two  arguments,  its  easily  shown  that  the  M  PAM  processes  ia^(t)}  are 
jointly  T-CS.  Furthermore,  as  pointed  out  in  Chapter  II,  these  PAM 
representors  have  constant  means  and  variances. 

This  representation  would  be  even  more  interesting  if  the  PAM 
processes  {ap(t)}  turned  out  to  be  jointly  WSS.  Using  Eq.  (3-23)  it  can 
be  shown  that  this  will,  in  fact,  be  the  case  if  and  only  if  the  Fourier 
transform  of  w— rather  than  w  itself— is  a  gate  function  (on  the  interval 
[-1/2T,1/2TJ).  With  such  a  choice  of  w,  we  obtain  the  "harmonic  series" 
representation  which  is  defined  and  discussed  at  length  in  Section  3. 

c)  Implementation  of  the  process-resolution  operation.  Knowledge 
of  Eq.  (3-12)  in  Theorem  (3-1)  allows  us  to  explicitly  specify  and,  in 
fact,  implement  the  process-resolution  operation;  i.e.,  the  operation 
which  resolves  the  random  process  x  into  its  representors  {a^}.  The 
operation  indicated  by  Eq.  (3-12)  has  two  realizations— both  of  which 
are  made  causal  [2]  by  the  incorporation  of  a  delay  of  length  T:  the 
multiplier- integrator  realization  of  Fig.  (3-1),  and  the  filter-sampler 
realization  of  Fig.  (3-2). 

The  multiplier-integrator  resolution  device,  as  shown  in  Fig.  (3-1), 

is  useful  only  for  computing  a  single  representor  random  variable,  since 

the  integrator  must  be  "dumped",  after  rnmuitation  of  a  ,  before  it  can 

np 

be  used  to  compute  a j )p*  But  this  can  be  easily  remedied  since  an 
effect  which  is  equivalent  to  dumping  can  be  obtained  by  following  the 
sampled  output  of  the  integrator  with  a  feedback-delay  as  shown  in 
Fig.  (3-3).  Furthermore,  if  the  sampler  at  the  output  of  the  integrator 
is  replaced  with  a  sample  -and-hold  device  (where  the  sample  value 
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obtained  at  tiae  nT  is  held  until  tine  (n+l)T  so  that  the  sampler's 

output  is  piecewise  constant),  as  shown  in  Fig.  (3-4),  then  the  output 

of  the  feedback-delay  device  is  the  continuous  representor  (PAM  process) 

&p(t)»  and  its  sequence  of  sample  values  is  the  discrete  representor 

(a  >  «  (a(nT)}. 
np  p 

Now,  using  M  of  these  resolution  devices,  we  can  construct  a 
causal,  linear,  periodically  tine-varying  system,  as  shown  in  Fig.  (3-5), 
which  resolves  the  process  x  into  its  representors.  Furthermore,  we  can 
reconstruct  x  fron  these  representors  with  the  causal,  linear, 
periodically- time-varying  system  also  shown  In  Fig.  (3-5). 

In  the  event  that  the  {^}  are  complex  and  indexed  from  -(M-l)/2 
to  (M-l)/2  such  that  »  ♦  ,  then  the  resolution-reconstruction  system 
of  Fig.  (3-5)  can  be  equivalently  realized  using  only  real  devices  as 
shown  in  Fig.  (3-6).  This  system  computes,  separately,  the  real  and 
imaginary  parts  of  the  representors. 

Unlike  the  multiplier- integrator  device  of  Fig.  (3-1),  the  filter- 
sampler  device  of  Fig.  (3-2)  can  be  used  without  modification  to 
continuously  compute  representors,  so  that  the  composite  resolution- 
reconstruction  system  takes  the  form  shown  in  Fig.  (3-7). 

d)  Examples. 

i)  Harmonic  translation-series-representations.  If  we  choose  as  our 
CON  set  in  L2(0,T]  the  complex  exponentials 

$_(t)  *  ~  ejZ,,pt/Tw(t) 

P  /f 

where  w  is  the  gate  function  defined  in  Eq.  (3-19),  then  the  periodic 

extension  of  $  is  just 
P 


* (t) .  i  ^tn 
*  Sr 


and  the  PAM  representation  of  Eq.  (3-19)  becomes: 

xo .  i  rn  *A<)'i2mn 

ff  p*(M-l)/2  P 

where  the  p^  PAM  process  is  given  by 
*«(*)  *  l  armw<t-nT) 


n 


np 


and  has  pulse-amplitudes  given  by 


anp  /  x(t*nT)e"^2t,pt/Tdt. 

For  this  harmonic  representation,  the  multiplier  function  employed  in 
the  resolution- reconstruction  system  shown  in  Fig.  (3-6)  are  simply 
sinusoids  given  by 

(t))  »  -  cos(2apt/T) 

P  /f 

Im$  (t) )  -  ~  sin (2cpt/T) . 

P 


Note  that  if  M  «  1,  then  the  process  x  is  itself  a  PAM  process  with 
full  duty-cycle  rectangular  pulses  and  pulse-amplitudes  (x(nT)). 


ii)  Walsh  translation-series-representations.  An  interesting  choice— 
from  the  point  of  view  of  implementation— for  our  CON  set  in  L2[0,T]  is 
the  set  of  two-level  Walsh  functions  [2]  defined  as 

1//F,  Hcl 


6  (t) 
p 


-1//F,  V-  t  e  (0,T)  -  I 


where  I  is  the  set  of  disjoint  intervals  contained  in  [G,T]  over  which 
P 

♦  is  positive.  The  first  few  sets  are 
F 
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Ix  *  [0,T] 

12  «  [O.T/2] 

13  «  [0,T/4],t3T/4,T] 

14  «  [0,T/4] , [T/2,3T/4] . 

Now,  since  multiplication  of  a  process  by  a  Walsh  function  is  equivalent 
to  amplitude  scaling  by  the  factor  1//T  followed  by  a  series  of  polarity 
reversals,  then  the  multipliers  shown  in  the  representation  system  of 
Fig.  (3-5)  can  be  implemented  as  polarity  reversal  switches  which  are 
actuated  at  the  transition  times  of  the  Walsh  functions  (end  points  of  the 
intervals  in  the  sets  1^).  Hence,  the  overall  implementation  of  the 
resolution-reconstruction  system  can  be  very  simple. 

Note  that  this  Walsh  representation  (Eq.  (3-20))  is  composed  of  the 
sum  of  M  piecewise  constant  processes,  and  is  therefore  also  piecewise 
constant  (except  in  the  limit  when  M  «  »). 

iii)  Karhunen-lobve  translation-series- representation.  The  basis 
functions  chosen  for  our  first  two  examples--sinusoids  and  two-level 
functions--are  appealing  because  of  the  resultant  convenience  in 
implementation  of  the  multiplier  function  generators  and/or  the  multiplier- 
devices  used  in  the  resolution  and  reconstruction  systems.  However,  there 
is  another  criterion  which  brings  to  our  attention  a  third  choice  for 
basis  functions.  Consider  the  choice  of  a  set  of  basis  functions  which, 
for  a  given  process  x  and  any  given  order  of  approximate  representation 
M,  minimises  the  integral -mean -squared  difference  between  the  process  x 
and  its  approximate  representation  x'.  In  this  case,  our  choice  would  be 
appealing  from  the  point  of  view  of  efficiency. 
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Now,  it  is  well  known  [2,25]  that  the  set  of  orthonoraalized 
eigenfunctions  p  *  1,2,..., Ml,  corresponding  to  the  M  largest 
eigenvalues  {X^>  of  the  linear  integral  operator  on  L2[0,T]  with  kernel 
k^Ct.s),  is  the  unique  set  which  ainiaizes  the  int  egra  1  -aean  -  squared 
difference: 

T  M 

J  E< txC')  -  JlVpt,))2)dt-  ‘op  '  (,,VL2- 

The  series  representation  which  eaploys  these  eigenfunctions 
H 

x'(t)  *  l  anDMt)»  1  e  t°»Tl 

pal  ™  P 

is  referred  to  as  the  Karhunen-Lohve  representation  for  x  on  [0,T]  and 
is  the  optiaua  (most  efficient)  discrete  representation.  The  eigenfunctions 
are,  by  definition,  the  solutions  to  the  integral  equation: 

T 

An*n(t)  »  J  k  (t,sH(s)dst  HtfOJ],  (3-25) 

F  F  q  AA  F 

and  if  x  is  T-CS,  then  they  are  also  the  solutions  to  the  integral 
equation: 

(n+l)T 

X*(t  nT)  -  /  k  y(t,s)*(s-nT)ds,  V  t  e  lnT,(n+l)Tl(3-26) 

P  P  nT  **  P 

Hence,  the  series  representation 
M 

x'(t)  «  l  a  *  (t-nT),  e  [nT,(n+l)T] 

p-1  np  p 

is  also  the  Karhunen-Lobve  representation  for  x  on  [nT,(n+l)T]  so  that 
the  resultant  TSR 

®  M 

x*  (t)  «  l  V  *n*n(t-nT).  Vtc  (-«,“) 

na-»  |i=l  np  p 
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minimizes  the  integral -mean-squared  difference  in  every  interval 

[nT, (n+l)T],  and  is  therefore  the  most  efficient  TSR  for  x. 

For  this  Karhunen-Lofcve  TSR,  the  matrix  A.  with  elements  {A^} 

“0  0 

is,  as  expected,  diagonal;  but  the  aatrices  (A  ;  r  t  0)  are  not,  in 

*T 

general  diagonal.  However,  there  exists  an  interesting  class  of  auto¬ 
correlation  functions  for  which  the  latter  aatrices  are  diagonal  and,  in 
fact,  scalar  multiples  of  A^.  For  details,  see  Theorem  (3-3)  and  the 
following  discussion. 

We  now  present  explicit  formulas  for  the  eigenfunctions  and 
eigenvalues  employed  in  the  Karhunen-Lofeve  TSR  for  two  types  of  CS  processes: 
(1)  Karhunen-Lofeve  TSR  for  the  video  signal:  In  Section  3  of  Chapter  II, 
we  presented  a  model  for  the  random  video  signal  which  included  a  formula 
for  the  autocorrelation  function  consisting  of  three  factors: 

kxx(t,s)  •  k1(t-s)k2(t,s)kJ(t,s). 

If  we  ignore  the  frame -to -frame  correlation  (because  of  its  relatively 
low-frequency  nature)  by  letting  in  Kq.  (2-26)  equal  1  (and  l-*»), 
then  kj(t,s)  *  1  for  all  t,s  and  the  video  signal  becomes  CS  with  period 
T  rather  than  LT.  Furthermore,  on  the  square  t,s  e  (O.T),  the  line-to- 
line  correlation  factor  k^t.s)  =  1  (from  Eq.  (2 -25)).  Thus,  we  have 

kxx(t,s)  *  kj(t-s),  Vt,s  e  (0,T] 

so  that,  from  Eq.  (2-24), 

kxx<t,S)  ‘  "i*"5'  •  Vt.scIO.T). 
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Now,  from  Eq.  (3-25) ,  our  TSR  basis  functions  are  the  normalized 
solutions  of  the  eigenfunction  equation: 


yp(t)  •  #/  •}t',|*,c*w*. 

and  are  given  (unnornalized)  by  [2] 


e  [0,T] , 


♦p(t) 


cos(*fo(t-T/2)yp)  ,  p  odd 
sin (wf0 (t-T/2) yp)  ,  p  even 


and 


1/irf 


o 


1  ♦  Y2 
P 


(3-27) 


|  >/y 

p  odd 

v-Ypt 

p  even 

where  (y  }  are  the  solutions  of 

r 

tan(trf0TYp)  « 

fo 5  i  in(i/Pj)> 

(2)  Karhunen-Lobve  TSR  for  the  time-division-multiplexed  signal: 

In  Section  2  of  Chapter  II,  we  presented  a  model  for  the  time-division- 
multiplex  (Tb^l  of  M  random  signals  {y^} .  If  we  assume  that  these  M 
component  signals  are  unccz related  and  NSS  with  autocorrelation  functions 
(kp),  then  the  composite  autocorrelation  function  for  the  TDMj  signal  x 
becomes: 


»  M 

kft.s)  *  l  l  k  (t-s)w  (t-nT)w  (s-mT) 
n,ra»-«  p*l  *  p  ^ 


(3-28) 


where  w  is  the  gate  function 
P 


wp(t)  - 


1, 

0, 


V  t  e  ( (p-l)T/M,pT/M]  -  Tp 
V-  t  t  T 
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so  that',  on  the  square  t,s  c  [0,T],  we  have 

M 

kxx(t,s)  *  I  Ut-sjw  Wn  ($),  ¥•  t,s  e  [0,TJ, 

p*l  r  r  i* 

and  on  the  square  t,s  e  T^,  we  have 


kxx(tl!)  ■  kp(t-s) • 


¥  t,s  e  T 


Now,  the  most  convenient  way  to  obtain  a  TSR  for  x  is  to  represent 
x  separately  on  each  of  the  M  component  intervals  {T^}  with  an  Ntk  order 
Karhunen-Lobve  representation: 


x’(t)  »  1  aV(t) 


V- 1  c  T  . 
P 


This  will  be  equivalent  to  representing  each  of  the  M  component  processes 
{y^}  individually: 


y'Ct>  ■  l  ®t), 


P«CTP. 


The  result  is  the  composite  (M*N)  order  TSR: 

ty  (0 

X'(t)  =  X  I  l  v-te  (-«,•). 

p«l  q»l  n— »  nq  q 


(3-29) 


If  the  component  processes  {y^}  have  exponential  autocorrelation 


functions 


.  .  -2irf  jt-s 

kp(t-s)  =  e  p*  ; 

then  the  eigenfunctions  {$]?}  can  be  obtained  from  the  formula  given  in 
the  Video  example.  Another  interesting  type  of  autocorrelation  function 
which,  unlike  the  exponential  type,  is  duration  limited,  is  that  which 
takes  the  form 


i 


I 
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k(t-s)  - 


[(t-svrjnu(s-t)*(s-t*TJnu(t-s)J,|t-s|<T«,  n  odd 

Tl 


0,  jt-s|  >  T*  , 

» 

and  has  the  shape  shown  in  Fig.  (3-8).  ‘ 

The  eigenfunctions  on  the  interval'  (-To/2,Tq/2]  (centered  for  convenience) 
which  correspond  to  this  type  of  kernel  are  very  easy  to  solve  for 
provided  that 


T'  <  t;/2. 

o  * 

The  approach,  like  that  used  to  obtain  the  eigenfunctions  for  exponential 
kernels  [2],irelies  on  converting  the  integral  equation  Eq.  (3-25)  to  a 
differential  equation.  For  the  above  type  of  kernel,  the  differential 

l 

equation  is: 

i  • 

,y  ■  ‘2tSislvt). *  1  £  n 

For  the  case  n  «  1,  the  kernfel  (autocorrelation  function)  is  triangular, 

i  .  • 

and  the  eigenfunctions  are  given  by:  ■ 


$p(t)  ■  cos(wpt). 


and  the  eigenvalues  by 


Ap  =  2k(0)/T^, 


wherp  the  {w  )  are  the  solutions,  of 


‘“V*’  ”  »!t~  -  r  u/(»  t  )2*ii 

r  poo  r  : 


T 


1/2 


. (3-30) 


e)  Finite  order  translation  series  representations .  The  TSR's 
established  in  Theorea  (3-1)  are  generally  applicable  to  all  CS  processes 
with  bounded  autocorrelation  functions,  but  will  usually  be  of  infinite 
order  (M  »  •).  There  are,  on  the  other  hand,  a  number  of  subclasses  of 
CS  processes  which  admit  special  finite  order  TSR's  which  are  not  based 
on  some  CON  set  in  L2[0,T]  as  in  Theorea  (3-1).  We  present  two  such 
subclasses. 

i)  Band United  cyclostationary  processes.  In  many  analyses  dealing  with 
randoa  signals  (particularly  in  conaunications  studies)  all  signals  are 
considered  to  be  bandlinited  to  the  interval  of  frequencies  [-fQ,fflJ 
where  f  is  the  highest  frequency  which  the  systems  (in  the  problem  of 
interest)  will  respond  to.  Thus,  given  f  and  a  T-CS  process  x,  we 
choose  the  smallest  integer  M  such  that  M/2T  >  fQ,  and  then  remodel  our 
process  as  its  bandlimited  (to  (-M/2T,M/2TJ)  version: 


x'(t) 


/  x(t) 


sin((t-T)TtM/T)  . 
(t-f)irM/T 


Now,  from  the  sampling  theorem  for  nonstationary  processes  (Theorem  (2-2)), 
we  have  the  following  mean-square  equivalent  representation  for  any 
process  x  which  is  bandlimited  to  f-M/2T,M/2T) : 


urn  T  WmT/tn  sin((t-raT/M)7iM/T) 
(  "  '  “Tt-raWnWl 


Thus,  if  we  define  the  M  basis  functions  {$  }  as 

P 


♦P(‘>  -  < 


M  a/2  sin((t-pT/M)*M/T) 
T  1  (t-pT/M)nM/T 


(3-31) 


and  the  M  representor  sequences 


{a  1  as 
np 
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*«p  *  (  »  )1/2  x(nT  +  PT/M)’  (3-32) 

then  we  can  rewrite  the  above  sampling  representation  as  an  IIth  order  TSR 
•  M 

x(t)  «  I  I  • ♦_(t-nT)  (3-33) 

n*-**  p*l  *  * 

where  the  C^(t-nT)}  are  orthonoraal  on  L2  {-•*,•]  (as  in  Eq.  (3-10)), 
and— analogous  to  Eq.  (3-12)  of  Theorea  (3-1)— the  elenents  of  the  M 
jointly  MSS  sequences  are  given  by 

m 

»np-  /  x(t*nT)fp(t)dt.  (3-34) 

ii)  Degenerate  cyclostationary  processes.  Me  use  the  tern  "degenerate" 
to  denote  those  randoa  processes  which  have  a  countable  (cardinality  equal 
to  that  of  the  set  of  natural  numbers)  ensemble  of  realizations.  We 
consider  here  a  specific  subclass  of  degenerate  CS  processes  which  are  of 
much  practical  interest  in  communications:  synchronous  M-ary  signals  of 
the  form 

00 

x(t)  =  l  ♦(t-nT.b  )  , 
n*-» 

where  4  is  a  deterministic  function  and  (bn)  is  an  M-ary  random  sequence 

where  each  random  variable  bR  has  the  M-ary  alphabet  of  realizations 

{oj ,c»2» . . . In  Section  4  of  Chapter  II,  we  defined  this  type  of  CS 

process  and  gave  six  specific  examples  including  frequency-shift-keyed 

signals,  pulse-position-modulared  signals,  and  others. 

th 

Now,  such  processes  can  be  re-expressed  as  an  M  order  TSR  as  in 
Eq.  (3-33)  where  the  pt*1  basis  function  is 
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♦p(t)  «  ♦(t,«p). 


and  where  the  M  jointly  WSS  sequences  (a^)  comprise  a  random  indicator 

sequence.  That  is,  for  each  n,  the  realizations  of  all  but  one  of  the 

M  elements  {a^  .a^, . . .  .a^}  are  zero,  and  the  non-zero  realization  is 

equal  to  1.  For  this  TSR,  the  matrix  of  correlation  sequences, 

A?  -  E{a  a*  ),  is  a  matrix  of  joint  probabilities  of  which  the  n-mth 
n-n  np  mq 

element  of  the  pq***  sequence  is  the  joint  probability  that  bR  *  o^  and 

b  «  o  .  Furtheraore,  the  aean  value  of  the  random  variable  a  _  is  just 
m  q  np 

the  probability  that  b^  •  ap. 


f)  Correlation  matrix  decomposition  and  solution  of  integral  equations. 
In  this  subsection,  we  show  how  the  TSR  for  an  autocorrelation  function 
can  be  used  to  obtain  the  solution  of  a  linear  integral  equation  whose 
kernel  is  the  autocorrelation  function,  and  we  show  how  to  simplify 
this  solution  through  decomposition  of  the  correlation  matrix  of  Eq.  (3-5). 


i)  Integral  equations.  The  solutions  to  many  problems  in  functional 
analysis  (including  estimation  and  detection  problems  as  discussed  in 
Chapter  IV)  can  ultimately  be  expressed  in  terms  of  the  solution  to  a 
linear  integral  equation--typically  a  Fredholm  equation  of  the  first 
kind  [29,31] 

/  k(t,s)h(s)ds  *  g(t),  V  t  e  S,  '3-35) 

S 

or  of  th('  second  kind 


/  k(t,s)h(s)ds  ♦  \h(t)  =  g(t),  V-  t  e  S, 
S 


(3-36) 
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or,  aore  generally, 

/  k(t,s)h(s,t)ds  «  g(t,t),  ¥-  t,T  e  S,  (3-37) 

S 

or 

/  k(t,s)h(s,T)ds  ♦  Ah(t,x)  «  g(t,T),  ¥■  t,t  e  S,  (3-38) 

S 

where  S  is  some  fixed  indexing  interval. 

Since  the  first  three  equations  can  be  considered  to  be  special  cases 
of  the  fourth,  we  will  restrict  our  discussion  here  to  this  fourth  equation. 

There  are  various  sets  of  circuastances  under  which  explicit  solutions 
to  this  integral  equation  can  be  obtained.  For  exaaple: 

(1)  If  S  ■  (-*•»•),  and  if  the  kernel  k  and  right  aeaber  g  are  tiae- 
invariant  (k(t,s)  *  k(t-s)),  then  the  integral  is  a  convolution  and  the 
solution  is  (foraally): 

h(t.s)  -f‘  {  jfj),., 

where  F" 1 ( • ) *.  is  the  inverse  Fourier  transfora  evaluated  at  t-s,  and 

•**s 

G,K  are  the  Fourier  transforas  of  g,k. 

(2)  If  the  kernel  k  and  right  aeaber  g  are  "separable  of  order  M": 

k(t,s)  -  l  A  a  (!)♦  (s) 
p,q»l  M 

M 

g(t,s)  »  l  C  ♦  (t)^  (s)  V  t,s  c  S 
p,q«l  M 

(Up)  can  always  be  chosen  orthonoraal  by  proper  choice  of  the  aatrices 
A,G),  then  the  integral  transformation  is  degenerate,  and  the  solution 
is  (formally): 
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h(t'3)  V  ,1  Wp(t)V5) 

where  the  matrix  B  is  given  by 
B  *  [A  ♦  XI] -lC 

where  l_  is  the  M  x  M  identity  matrix. 

Note,  if  S  is  finite,  then  under  a  fairly  liberal  set  of  constraints  on 
k,g.  arbitrarily  close  separable  approximations  to  k,g  can  be  obtained 
[34,36]. 

Now,  as  a  combination  of  these  two  methods,  we  present  the  following 
new  (to  our  knowledge)  formal  solution  to  Eq.  (3-38): 


THE0REM(3-2):  If  the  symmetric  functions  k,g  are  each  jointly  T-periodic 
in  their  two  arguments  (k(t+T,s+T)  *  k(t,s)),  and  if  the  section 
functions  k(«,t),k(t, •).  g(**t).g(t,*)  are  in  L2[0,T]  for  every 
t  e  (-*,“),  then  the  solution  to  the  integral  equation 

m 

/  k(t,s)h(s,f)ds  +  Xh(t,r)  *  g(t,i)  V-  t , t  c  (-*»,«•)  (3-39) 

~eo 


has  the  representation: 


w  W 

h(t,t)  -  l  l  B^m  ♦  (t-nT)4*(T-»T) 


(3-40) 


p,q»l 


where  (4  }  is  any  CON  set  on  L2[Q»T],  and  the  sequence  of  matrices  (Bj 


is  given  by  the  inverse  z-transform 

1/2T  i2nrTf 
B  3  T  /  B(f)ej2  rTfdf 
r  -1/2T  ~ 


(3-41) 


where  the  matrix  of  functions  B(f)  is  given  by 
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B(f)  -  [A(f)  ♦  Uy'Gtf) 


(3-42) 


where  the  matrices  A,  G  are  the  z- transforms 


A(f)  »  l  a  e'j2lrTf 
G(f)  «  ?  G  e~l2nrTf 


(3-43) 


r«-» 


and  where  the  elements  of  the  sequences  of  matrices  A  ,  G  are  given  by 

-r  -f  ,  * 


T 

A?1  •  //  k(t*rT,T)**(tU  (t)dtdx 
r  0  Pm 

T 

«  //  g(t+rT,x)**(t)4  (x)dtdT.  (3-44) 

0  F  M 


Proof:  Employing  the  completeness  relation  (Eq.  (3-9))  as  in  the  proof 
of  Theorem  (3-1)  (Eq.  (3-15))  yields  the  translation  series  representations 

•  «> 


k(t.s) 


1  l  l 

n,m*-«  p,q*l 


g(t,s)  -  l  l  Gjq  6  (t-nTH*(s-mT).  (3-45) 

n,»  p,q  ‘  f  m 

Now,  assuming  the  form  Eq.  (3-40)  for  h,  and  substituting  these 
representations  into  Eq.  (3-39),  and  employing  the  orthonorraality  of  the 
(♦p(f’^T)}  (Eq.  (3-10))  yields  the  discrete  version  of  our  integral 
equation  Eq.  (3-39): 


1 1 .  c.  c  *  ■  «r  - 


4*1  m*-® 


(3-46) 


But,  the  first  term  it.  the  left  member  is  simply  the  composition  of  a 
discrete  convolution  and  a  matrix  product.  Recognizing  this  and  taking 
the  z-transform  of  both  sides  of  this  equation 
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yields 

A (f ) •  B(f )  ♦  AB(f)  -  G(f) 
so  that  (formal ly) 

B(f)  «  IA(f)  ♦  AIJ^Gff). 

Now,  the  sequences  {B^>  are  the  inverse  z-transforns  given  in  Eq.  (3-41). 

QED 

Note  that  the  matrix  B(f)  which  characterizes  the  solution  given  in 
this  theorem  is  defined  in  terms  of  the  inverse  matrix  [A(f)  ♦  At)“l 
where  A(f)  is,  in  general,  infinite  dimensional.  Thus,  our  solution  is, 
in  general,  only  formal.  However,  in  the  event  that  finite  order  (say  M) 
translation  series  representations  for  k,g  exist,  then  the  matrices 
A,  B,  G  are  all  of  dimension  M  x  M  and  the  solution  can  be  of  great 
value  as  illustrated  in  Chapter  IV.  Furthermore,  even  when  A(f)  is 
infinite  dimensional,  the  inverse  (A(f)  ♦  AJJ"1  can  be  computed  provided 
A(f)  is  either  diagonal  or  can  be  appropriately  decomposed  into  the  sum 
of  a  diagonal  matrix  and  a  finite  rank  matrix.  In  this  next  subsection, 
we  discuss  the  decomposition  of  A(f)  and  give  a  number  of  examples. 

However,  before  turning  to  this  topic,  we  note  that  the  solution 
method  of  Theorem  (3-2)  can  easily  be  extended  (as  shown  in  Chapter  IV) 
to  the  case  where  the  basis  functions  {4^}  are  neither  orthonormal  nor 
duration  limited  to  (0,T). 
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ii)  Correlation  Matrix  decomposition.  In  the  previous  section  on 
integral  equations,  we  saw  (Theorem  (3-2))  that  an  operator  equation  of 
the  form 


[A  -  X|]  «B  *  G,  (3-47) 

where  A,  B,  G  are  operators  on  the  function  space  L2 (-•»,»)  with  kernels 
k,h,g  which  admit  a  TSR  of  order  M  (Eq.  (3-45)): 

*(t,s)  -  l  £•(  t-nT)A4*(s-»T) 

n,m«-» 

(where  £  is  an  M-vector,  an  M  x  M  matrix),  can  be  reduced  to  an 
operator  equation  of  the  form 

[A(f)  ♦  XI]*B(f)  «  G(f)  (3-48) 

where  A,  B,  G  are  M  x  M  matrices  of  periodic  functions : 

»<f>  ■  I  A.e'JJ"rTf:  (3-49) 

and  that  the  solution  to  Eq.  (3-47)  can  be  obtained  from  the  solution  to 
Eq.  (3-48): 

B(f)  «  [A(f)  ♦  U]-lG(f),  (3-50) 

when  the  indicated  matrix- inverse  exists. 

We  now  present  three  types  of  correlation  matrices  A(f)  where  this 
inverse  not  only  exists,  but  also  is  relatively  easy  to  compute- -even  for 
M  =  «.  First,  however,  we  mention  that  since  A,  G  (and  therefore  B)  are 
1/T-  periodic,  then  the  matrix  inversion  need  only  be  curried  out  for 
values  of  f  e  [-1/2T.1/2TJ . 
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(1)  Infinite-dimensional  diagonal  matrices:  We  begin  with  a  theorem. 

THEOREM (3-3) :  If  A(f)  is  the  correlation  matrix  for  the  kernel  k 
(defined  by  Eqs.  (3*43,44)  of  Theorem  (3-2)),  and  if  k  has  the  wide-sense 
Markov- like  property: 

k(t+rT,r)  «  ork(t,T)  ¥•  t,T  e  [0,T],  ¥r 

for  some  sequence  (or),  then  A(f)  decomposes  into  the  product  of  a  1/T- 
periodic  function  and  a  constant  matrix 

A(f)  *  *<>  l  a  e‘j27rrTf, 

r 

and  if  the  CON  set  of  basis  functions  {$  )  employed  in  the  TSR  for  k  are 

P 

the  eigenfunctions  of  the  operator  A  with  kernel  k,  then  Aq  is  the  diagonal 
matrix  A  whose  non-zevo  elements  are  the  corresponding  eigenvalues. 
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There  are  several  CS  random  processes,  of  interest  to  us  in  this 
thesis,  whose  autocorrelation  functions  k  satisfy  the  hypothesis  of 
this  theorem.  For  the  sake  of  continuity,  we  give  as  two  examples 
those  processes  for  which  the  eigenfunctions  were  obtained  in  Subsection 
2d: 

(la)  Random  video  signal:  From  our  model  developed  in  Chapter  II 
(Eqs.  (2- 23; 24, 25))  we  have 

k(t+rT,T)  ■  c*rk(t,r)  V-  t,T  e  [0,T] 

■  v!M 

where 

«  ,  |(r)aodl|  |r|-|r|modL 
r  v2  M3 

and  p2,  p»  are  the  line-to-line  and  frame- to- frame  correlation  factors, 
and  L  is  the  number  of  lines  per  frame.  So,  from  Theorem  (3-3)  we  have 
the  inverse  matrix 

D(f)  -  (A(f)  ♦  uyl 


with  elements 


6 

‘w  *  x .  *  i  • 

p.  r 
r 

where  the  { A  )  are  given  by  Eq.  (3-27),  and  if  we  ignore  frame-to 

P  Irl 

frame  correlation  (pj  *  1,  then  ar  =  p£  1  and 

i  a  c'j2"rTf - i id - . 

r=-«  r  (1-P2)2  +  4p2sin2(«Tf) 


(3-51) 


(3-52) 


(lb)  Time-division-multiplexed  signal:  As  discussed  in  Subsection  2d 
it  is  aost  convenient  to  obtain  the  TSR  for  the  TOM^  signal  in  tens  of 
M  Karhunen-Loeve  expansions — one  for  each  component  signal  being  multi¬ 
plexed.  Thus,  the  composite  correlation  matrix  is  composed  of  N 
matrices  (the  M2  -  M  off-diagonal  matrices  are  zero  because  the  M 
signals  arc  assumed  uncorrelated) 


A(f) 


with  elements 


A1  (f) 


A2(f) 

'  AM(f) 


,1#«  -  v  »  -j2irrTf 


A(f)  ■  l  Ar  t  e 


where 

T/M 

■  //  k,(t+rT-T)^(t+(t-l)T/M)*/(f(t-l)T/M)dtdT. 

0  •  H 

But,  for  the  class  of  correlation  functions  discussed  in  Subsection 
2d:  namely 

-~-t  (t-Tvr4)"u(T-t)+  (T-t+Tt),,u(t-T)] ,  I  t-T I <T£,  n 


k^t-r)  » 


0, 


t-T  >  T., 


we  have,  for  T/2M  <  T£  <  T(l- 1/M) , 


k£(t+rT-t)  «  ark£(t-i) 


V  t,T  e  (O.T/MJ 


1,  r  =  0 
0,  r  t  0 


odd 


where 


Hence, 


and 


and 


«  x  a  6  , 
r,l  P  r  pq* 


A*(f)  «  A4, 


A(f) 


L1 

0 


L*  s. 


so  that  the  inverse  matrix  given  by 


D(f)  «  (A(f)  ♦  XI I 


-l 


Q.1 

0 


•B.M 


has  elements 


d* 

W  X  ♦  A* 

P 

where  the  {A4}  are  given  by  Eq.  (3-30), 


(3-53) 


(2)  Infinite  dimensional  diagonal-plus-finite-rank  matrices:  Again,  we 
begin  with  a  theorem. 


US 


THE0REM(3-4) :  If  the  inverse  matrices  D  *  [£  ♦  RS ' J " 1 ,  ,  and 

F'1  •  [I  ♦  S'S'1^]"1  all  exist,  then 

D  «  2"1  -  (3-54) 

A  proof  of  this  theorea  (known  as  the  Matrix  Inversion  Lemma) 
can  be  found  in  the  appendix  of  reference  [49]. 

Our  application  of  this  theorea  is  to  the  case  where  the  infinite¬ 
dimensional  correlation  matrix  A(f)  can  be  decomposed  into  the  sua  of 
an  invertible  diagonal  matrix— call  it  A^— and  a  finite  rank  (say  N) 
matrix  RS 1 ,  where  R,S  are  infinite-by-N  dimensional  aatrices.  For  then 
the  infinite-dimensional  matrix  inverse  D(f)  »  (A(f)  ♦  XI J”1  can  be 
computed  in  terms  of  the  inverse  of  the  N  x  N  matrix 
F(f)  «  H  ♦  S'(fJ*[A  ♦  AI]*lR(f)]  by  using  Eq.  (3-S4)  of  Theorem  (3-4) 
where  £  *  A  ♦  X^. 

We  present  a  major  class  of  CS  processes  whose  infinite  dimensional 
correlation  matrices  can  be  so  decomposed,  and  then  we  present  a 
variation  of  that  class: 

(2a)  CS  processes  which  are  the  outputs  of  periodically  time-varying 
finite-dimensional  linear  dynamical  systems  driven  by  white  noise: 

Since  the  class  of  processes  which  urc  generated  by  passing  white  noise 
through  a  finite-dimensional  (order)  linear  dynamical  system  are  of  much 
interest  in  statistical  communication  and  control  theories  [49-51 ] 
(because  of  their  analytical  and  computational  tractability) ,  we  will 
present  our  result  on  decomposition  in  the  form  of  a  theorem  and  proof 
which  will  be  convenient  for  future  reference: 
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THE0R£M(3-S):  If  x  is  the  T-CS  output . process  of  a  T-periodic  N**1 

'*  i  ,  * 

order  linear  dynamical  system  driven  by  a  vector  (of  arbitrary  diaension) 

1  ,  * 

of  jointly  white  WSS  processes,  then  the  infinite  dimensional  correlation 
aatrix  (Eqs.  (3-43,44)),  corresponding  to  the  TSR  of  Theorem  (3-1),  for 
x  adaits  thp  decoaposition: 

A(f)  -  An  ♦  R(f)S*  (3-55) 

I  . 

where  R(f),  S  have  dimensions  »  x  2N,  and  if  the  set  of  basis  functions 
(4  }  employed  in  the  TSR  for  the  autocorrelation  function  k  are  the 

p  l  *'  |  XX 

eigenfunctions  of  the  operator  A  whose  kernel  is  k  ,  then  A0  is, the 
diagonal  matrix  whose  non- zero  elements  are  the  corresponding 
eigenvalues.  1  , 

i  th 

Proof:  Consider  the  state-equation  description  of  sn  N  order  linear 

•  \ 

dynamical  system:  1 


••F(t)£(i)  ♦  ;b(t)n(t) 


x(t)  =  c*(t)£(t), 


(3-56) 


where  £  is,  the  N  x  1  state  vector  of  random1  processes,  F  is  the  N  x  N 

I 

state  transition  matrix,  b  is  the  input  matrix,  n  is  the  input  vector  of 
jointly  white  WSS  processes  with  autocorrelation  matrix 


E{n(t)n' (x) }  »  R5(t-tj , 


(3-57) 


and  c'(t)  is  the  1  x  ?!  output  matrix,  and  x  the  output  process.  Now, 
the  steady-state  output  x  is  given  explicitly  by 
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t 

x(t)  -  /  c'  (t)#(t,T)b(x)n(T)dx  (3-S8) 

mm 

where  ♦  is  the  fundamental  solution  aatrix  corresponding  to  the 
transition  aatrix  £  [52].  Froa  this  egression  and  Eq.  (3-57),  it  is 
easily  shown  that  the  steady-state  autocorrelation  function  for  x  is 
given  by 


k^Ct.s)  »  £'(t)h(s)u(t-s)  ♦  fc’  (s)h(t)u(s-t)  (3-59) 

where  the  N  x  1  Matrices  g,  h  are  given  by 


&(t)  «  i(t»t0)c(t) 
t 

h(t)  «  /  i(t0,T)b(T)Rb»(x)*‘(t.T)dxc(t)  (3-60) 


and  t0  is  arbitrary  (we've  used  the  property  of  all  fundamental 
solution  Matrices:  *.(t,T)  3  ♦(t,to)*(tQ,t)  V-t0,t,t). 

Thus,  using  Eq.  (3-S9)  in  Eq.  (3-44)  yields  the  following  expressions 
for  the  elements  of  the  matrix  of  correlation  sequences: 


!  «;i(r)hqi(0), 
N  *«’• 

j.V'V0’- 


r  >  0 
r  »  0 
r  <  0 


(3-61) 


where 

a  T 

gpi (r )  -  J  gi(t+iT)4p(t)dt 
a  T 

h  •  (r)  -  /  h.(frT)4  (t)dt  , 

pi  0  p 
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and  g.,  h.  are  the  elements  of  the  g,  h  vectors  of  Eq.  (3-60). 

Mo v,  substituting  Eq.  (3-61)  into  Eq.  (3-43),  we  obtain  the  following 
expression  for  the  correlation  Matrix: 

A(f)  -  ♦  G*(f)iV  ♦  H*(f)G^,  (3-62) 

where  the  elements  of  the  •  x  N  dimensional  Matrices  G(f),  H(f)  are 

v®  *  Xwr>'i2'rT{ 

Vf)  • 

and  the  elements  of  the  •  x  N  dimensional  matrices  G  ,  H  are 

— o  — o 

(?pi (0) ) , thpi (°) ) •  Hence, 

A(f)  *  ^  ♦  R(f)S»  (3-63) 

where 

R(f)  -  [G* (f ) ,H* (f ) ] 
and 

l- 

Finally,  if  {^}  are  the  eigenfunctions  for  the  operator  with  kernel 
kxx’  then  as  in  Theore,n  (3"3)»  Aq  *  A. 

QED 
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(2b)  TDMj  processes  with  component  signals  which  have  been  generated 
by  finite-diaensional  tiae-invariant  systems:  As  a  variation  on  the 
above  result,  consider  the  IDMj  process  discussed  throughout  this 
section— but  this  tiae  consider  coaponent  processes  which  can  be  aodeled 
as  the  outputs  of  tiae-invariant  finite-diaensional  linear  dynaaical 
systems  (of  order  N).  As  in  the  previous  subsection,  the  coaposite 
correlation  matrix  takes  the  fora 


A(f) 


A*(f) 


A2(f) 


•AM(f) 


d  th 

where  Ap (f)  is  the  correlation  matrix  for  the  p  coaponent  process, 

and  by  Theorem  (3-5)  can  be  decoaposed  as  follows 

*p<«  -  *o,p  * 

where  R  ,  S  are  *  x  2N  dimensional  matrices,  and  if  the  basis  functions 
"P  “P 

used  in  the  TSR  are  the  eigenfunctions,  then  ^  »  tP,  the  diagonal 
matrix  of  eigenvalues.  Now,  since 


{A(f )  ♦  Ajy1  » 


(A^fJ+AIJ- 


[A2(f)^U 


-1 


*‘.(AM(f 


(5-64) 


then  this  composite  inverse  can  be  obtained  by  employing  Theorem  (3-4) 
to  obtain  the  inverse  of  each  of  the  M  component  matrices— the  total  . 
computation  requiring  the  inversion  of  M  matrices  of  dimension  2.N  x  2N. 
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(3)  Finite-dimensional  diagonal  and  diagonal-plus-rank-1  Matrices: 

Our  third  and  last  class  of  correlation  Matrices  to  be  considered  is* 
in  fact*  a  special  case  of  the  previous  two  classes.  We  consider  here 
correlation  Matrices  which  are  not  only  finite-diaensiou&l*  but  also 
either  diagonal  or  diagonal-plus-* *nk-l. 

If  A(f)  «  4(f),  a  diagonal  matrix  with  non-zero  elements  (Ap(f)), 
then  the  inverse  Matrix 


D(f)  -  (A(f)  ♦  Xll'1 


has  elements 


or 


if  A(f)  « 


(n  — . 

♦  a 

A(f)  ♦  R(f)S'(f), 


elements  (rp(f)),{sp(f)}*  then, 
D(f)  has  elements 


(3-65) 


where  R(f),  S(f)  are  M  x  1  vectors  with 
from  Theorem  (3-4),.  the  inverse  matrix 


r»<f>  Vf> 
h  (O  ♦  A  J  h  (if)  ♦  A  ^ 
- 

11  a7(7)  V  a  ) 


(3-66) 


so  that,  in  both  cases,  the  inverse  matrix  D(f)  is  given  explicitly 
with  no  need  for  further  computation. 

Now*  the  purpose  here  is  to  present  two  specific  exanples  of  CS 
processes- -one  of  which  has  a  finite-dimensional  diagonal  correlation 
matrix,  and  the  other  of  which  has  a  finite-dimensional  diagonal -plus- 
rank-1  matrix.  Both  of  the  examples  given  here  are  models  of  random 
signals  which  are  of  much  interest  in  communications: 
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(3a)  Time-division-multiplexed  PAM  signals:  As  an  alternative  to 
the  TDMj  and  TDMj  schemes  introduced  in  Chapter  II  for  time-multiplexing 
a  multiplicity  of  signals,  we  consider  here  the  TDM- PAM  scheme.  If  our 
M  component  processes  {y^}  are  bandlimited  to  [-1/2T.1/2T] ,  then  they 
can  be  periodically  sampled  every  T  seconds  without  loss  of  information. 
If  these  M  sample  sequences  are  now  interleaved  in  time  to  provide  a 
single  composite  sequence  of  samples --one  every  T/M  seconds— then  this 
composite  sequence  can  be  employed  to  amplitude-modulate  a  stream  of 
pulses  to  obtain  a  time-division-multiplexed  PAM  signal  x: 


x(t)  8  l  l  yo(nT)q(t-nT-pT/M). 
n«-*  p»l  p 


(3-67) 


If  the  M  component  processes  are  jointly  WSS,  then  x  is  a  T-CS  process 
with  an  Mth  order  TSR  with  a  matrix  of  correlation  sequences  given  by 


•  E<yp(nT)yq(.T)) 


so  that,  if  the  component  processes  are  also  mutually  uncorrelated,  then 

aP«  «  aPP  6  , 

n-m  n-m  pq 

and  the  correlation  matrix  A(f)  is  diagonal: 


A  (f)  -  «  l  APP  e-j2,,rTf  . 

pq  pm  ;  * 


(3-68) 


(3b)  Synchronous  M-ary  signals:  In  Subsection  2e,  we  pointed  out  that 
synchronous  M-ary  signals  such  as  FSK  admit  M**1  order  TSR's  where  the 
matrix  of  correlation  sequences  is  a  matrix  of  joint  probabilities: 
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A***  «  Prob[b  «  a  and  b  «  a  ]. 
n-»  n  p  m  qJ 

Now,  if  we  assume  that  the  random  variables  in  the  modulating  sequence 
{bn}  are  mutually  statistically  independent,  then 

AW  .  |  ProbIbn  *  VProb[b»  *  “,!• 

"  t  vrobtb»  ■  y-  » ■ « 

-  P(p)P(q)(l  -  tm)  ♦  «p,P(p)«ra 

where  P(p)  -  Prob(bn  «  opJ  for  all  n.  Hence,  the  M  x  M  correlation 
matrix  is  given  by 

Vf)  ‘  V(P)  *  P(P)P(,)  i„  ''J2"Tf  >  (3-69) 

and  is  composed  of  the  sum  of  a  diagonal  matrix  and  a  rank-1  matrix. 


g)  Implementation  of  periodically  time-varying  linear  systems. 

As  shown  in  the  previous  subsection,  the  TSR  of  an  autocorrelation 
i unction  which  is  the  kernel  of  a  1 inoar  integral  transformation  (operator) 
is  reflected  in  the  kernel  of  another  transformation  which  is  the  solution 
of  an  operator  equation  (Eqs.  (3-38,47)).  For  some  applications 
(especially  estimation  and  detection  as  discussed  in  the  following 
Chapter)  it  is  desired  to  physically  implement  the  solution  operator 
as  a  linear  system  whose  impulse  response  function  is  the  kernel. 

An  impulse  response  function  h  which  has  an  Mth  order  TSR 


»  M 

l  l 


n,m®-»  p,q=l 


bP<* 

n-ra 


$p(t-nT)$*  (t-mT) 


h(t.t) 


r 
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has  the  straightforward  implementation  shown  in  Fig.  (3-9) ,  where 
the  input  is  applied  to  a  bank  of  M  tine-invariant  filters  (with  transfer 
functions  {♦*})  whose  outputs  are  periodically  sanpled  every  T  seconds 
and  applied  to  an  M  x  M  matrix  of  time- invariant  sampled-data  filters 
[53,54]  (with  transfer  functions  (B  (f)}  which  are  z  transforms  of  the 
impulse  response  sequences  {B^}) .  The  sequences  of  output  samples  from 
the  sampled-data  filters  (assumed  weighted  sequences  of  impulses)  are 
then  applied  to  a  bank  of  M  pulse-generating,  time-invariant  filters 
(with  transfer  functions  (4^))  whose  outputs  are  summed  to  form  the 
final  output  of  the  system. 

Note  that  this  system  can  be  obtained  from  the  resolution- 
reconstruction  system  of  Fig.  (3-7)  simply  by  inserting  the  matrix  of 
sampled-data  filters  between  the  resolution  portion  and  the  reconstruction 
portion.  Similarly,  a  multiplier- integrator  type  cf  implementation 
for  the  system  with  impulse  response  function  h  can  be  obtained  from  the 
resolution-reconstruction  system  of  Fig.  (3-5)  by,  again,  inserting  the 
matrix  of  sampled-data  filters  between  the  resolution  and  reconstruction 
portions . 

Finally,  it  should  be  mentioned  that  the  impulse  response  function 
h  will  not,  in  general,  represent  a  causal  system,  so  that  a  physical 
implementation  must  incorporate  a  delay,  say  tQ,  so  that  h(t,i)  *  h(t-tQ,T) 
is  the  impulse  response  which  is  implemented.  Notice,  however,  that  if 
one  started  with  a  causal  h: 

h(t,T)  =  h(t,T)u(t-t) 


where  u  is  the  unit  step  function,  and  one  expanded  it  into  a  TSR,  then 


the  sampled-data- filter  impulse  response  sequences  would  satisfy 
T 

Cm  *  //  h(t+(n-m)T,T)^(t)^(x)dtdT 
«>  0  for  n-m  <  0. 

Hence,  the  sampled-data  filters  would  be  causal.  Furthermore  since 

the  output  filters  {$^}  are  causal,  and  the  input  filters  {♦*} 

can  be  made  causal  with  a  delay  of  length  T,  then  the  overall  (approximate) 

realization  (which  uses  only  a  finite  number  of  terms)  shown  in  Fig.  (3-9) 

would  be  causal  with  a  delay  of  t  *  T.  Note  that  a  finite  order  TSR 

o 

cannot  be  causal  without  the  incorporation  of  at  least  a  T-second  delay. 

h)  Generalized  Fourier  transforms  for  cyclostationary  processes. 

A  rather  disappointing  quirk  of  the  transfer  function  theory  for  linear 
time- invariant  systems  (as  it  stands)  is  that  it.  applies  only  to 
deterministic  signals,  so  that  random  input  ani  output  signals  cannot, 
in  general,  be  related  by  the  simple  formula 

X(f)  *  H(f)Y (f) 

where  X,  Y,  H  are,  respectively,  the  Fourier  Transforms  of  the  output  x, 
the  input  y,  and  the  impulse  response  function  h.  The  reason  being 
that  Pouiier  transforms  of  random  signals  are  not,  in  general,  defined. 

The  standard  argument  against  the  possible  existence  of  a  Fourier 
transform  of  a  stationary  random  process  is:  the  Fourier  transform- -like 
any  other  linear  transformation  of  a  random  process -should  be  interpreted 
as  the  ensemble  of  Fourier  transforms  of  the  deterministic  functions 
which  comprise  the  ensemble  of  realizations  of  the  process;  but  due  to 


the  "continuing"  nature  of  a  stationary  process,  its  realizations  will 
not  be  in  L2 (-«,•),  and  therefore  will  not  be  Fourier  transformable. 

The  prevalence  of  this  attitude  is  evidenced  by  this  statement  from  a 
popular  modem  engineering  textbook  on  random  processes  [Ref.  3, 
pp.  336,465]:  "We  assume  the  reader  is  familiar  with  linear  systems 
and  transform  techniques.  We  should  emphasize,  however,  that  we  transform 
only  deterministic  signals  and  not  stochastic  processes....  Stationary 
processes  have,  in  general,  no  Fourier  transforms." 

The  purpose  of  this  subsection  is  to  employ  the  TSR  of  Theorem  (3-1) 
to  define  a  simple,  yet  adequate,  generalized  Fourier  transform  for  the 
class  of  all  finite  mean-square  CS  (and  WSS)  processes.  With  this 
generalized  Fourier  transform,  the  transfer  function  methods,  which  are 
so  useful  in  the  analysis  of  linear  time-invariant  systems  and 
deterministic  signals,  can  be  directly  extended  to  the  analysis  of  these 
systems  and  random  signals.  For  example,  the  frequency-domain  input- 
output  relations  of  Chapter  II  (Eqs.  (2-7, 8, 9)),  which  were  derived  in 
a  round-about  way  in  the  time  domain,  can  be  re-derived  in  the  frequency 
domain  in  one  or  two  simple  steps.  This  is,  in  fact,  done  in  (Ref.  3, 
pp.  465-467]  by  "...ignoring  the  mathematical  difficulties." 

Wo  begin  by  presenting  a  generalized  version  of  the  well  known 
formal  derivation  of  the  generalized  Fourier  transform  of  a  periodic 
function.  Consider  any  periodic  function  x  in  the  Hilbert  space 
LP(T) (—"»*) *  de*inec*  *n  Sec.  (S'1)*  If  Up}  is  any  CON  set  in  L2[0,T], 
then  x  admits  the  deterministic  translation  series  representation 


159 


x(t) 


l  l 

p*l  n«-» 


Vp(t'nT) 


a.e.  t  e  (-•»,<*) 


*  I  l 

p  ® ;  p 


where 


a  ■  /  x(t+nT)$*(t)dt  ■  a, 
np  J  'V  OP 


¥■  n. 


Now,  we  define  the  general! ted  Fourier  transform  for  any 

x  e  (-»,»)  to  be  the  equivalence  class  of  transformations  Fl2(*} 
- LfJJ -  Lp 

with  representatives 


fL2{x(t)>  -  X(f)  $  l  a  l  FL2(^(t-nT)} 

P  p  w  n  F 

■  l‘0Pl  y«  e'J2roTf 

p  n  v 

•  f  l  aop  l  «p(«/T)S(f-./T) 

P  ® 

where  F,  • }  is  the  usual  Fourier  transform  defined  on  L2 (-»,«•) ,  and 

M 

where  the  last  equation  is  the  Poisson  sum  formula  [2,55].  The 
equivalence  relation  which  generates  this  equivalence  class  is 

00 

Xj  *  x2  <«>  /  jx,(f)  -  X2(f)|df  «  0, 

where  and  X2  are  the  two  transforms  of  x  corresponding  to  the  two 

representatives  of  F. 2  obtained  from  any  two  CON  sets  in  l2[0,T], 

Lp 

and  x  is  any  function  in  L2^.  It  is  not  difficult  to  show  that  all 

representatives  of  F.2W  corresponding  to  any  specific  signal  x,  do 

Lp 

indeed  satisfy  this  equivalence  condition. 


Now  consider  the  specific  CON  set  of  complex  exponentials 


VnT>  ■  ^  V 

Also,  the  TSR  coefficients  are 

/T  aQp  -  yix(t)e';,2wI>t/Tdt  £  ap 

and  are  the  familiar  Fourier  coefficients  for  the  harmonic  Fourier 
series  representation  for  x.  Thus,  our  generalized  Fourier  transform 
now  takes  the  well  known  form  [55] 

00 

X(f)  »  l  a  {(f-p/T) . 

p*-co  * 

This  generalized  Fourier  transform  can  easily  be  extended  from 
deterministic  to  random  periodic  functions,  as  discussed  in  [Ref.  3, 
p.  467],  simply  by  allowing  the  Fourier  coefficients  to  be  random 
variables.  In  fact,  our  more  general  definition  of  the  generalized 
Fourier  transform  F.  2  can  just  as  easily  be  extended  from  deterministic 

S’ 
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periodic  functions  to  cyclostationary  random  processes: 


DEFINITION:  The  generalized  Fourier  transform  for  any  randoa  process, 

I*  f~m  m! 

say  x,  which  is  contained  in  the  Hilbert  space  CS(T) _  *  _  is  defined 

to  be  the  equivalence  class  of  transformations  Fu  {•}  with  representatives 

CS 

00  00 

Vx(t,) ' X(f)  5  tl  „L 

*  i  ■ 

p  n  v  v 

where  {^}  is  any  CON  set  in  L2[0,T]  (extended  to  the  zero~function 
outside  [0,T])  and  {a^}  are  the  random  TSR  representors 
T 

anp  «  /  x(t+nT)$*(t)dt 

and  is  the  usual  Fourier  transform  defined  on  L2(-»,»). 

The  equivalence  relation  which  generates  this  equivalence  class  is 

00 

\  *  x2  <»>  //  ! E{ CXj  Cf ) -x2 (f D ) (Xj ( w) -x2 Cv) ) *  > Idfdv  .  0 

•on 


where  and  are  the  two  transforms  of  3  corresponding  to  the  two 

representatives  of  Fu  obtained  from  any  two  CON  sets  in  L2(0,T],  and 

HCS 

x  is  any  random  process  in  McS(T)^”*****^' 

That  this  relation  is  indeed  an  equivalence  relation  [2]  can  be 
verified  by  showing 

QO 

•//|E{(X1(f)-X2(f))(Xl(v)-X2(v))M|dfdv  *  0 


<=-*  IH  |x][(t)-x2(t)|2}  =  0  He  (-»,*), 


where  x^  and  x2  arc  the  random  processes  with  autocorrelation  functions 
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%*i(t'#)  *  F*F||l{E{Xi(f)Xi(v)))  1  *  1 »2 

where  F'F.J1  is  the  generalized  double  Fourier  transform  inverse  for 
T» 

periodic  functions  of  two  variables;  and  using  the  fact  that 


X1  *  X2  <B>  e^Ixi^"x2^I2^  *  0  *  t  e  (“"»•) 

is  a  valid  equivalence  relation. 

The  following  theorem  extablishes  the  equivalence  under  expectation 
of  our  generalized  Fourier  transform  for  CS  processes  and  our  generalized 
Fourier  transfora  for  deterministic  periodic  functions: 


THEOREM (3-6) :  The  generalized  Fourier  transforms^  ,  F^2  are  equivalent 
under  expectation  in  the  following  sense: 


E{Fh  { x (t ) }}  «  FjafEtxft)})  «  FL2{mx(t)) 
CS  P  P 


E(Fm  (x(t) }F*  (x(s)}}  =  F-F.2{E{x(t)  x*(s)}} 

"cs  MCS  LP 

■  ■  V**> 


for  every  x  e  hcs(t) 


Proof:  Let  X(f)  be  the  representative  of  the  equivalence  class 

Fh  {•}  corresponding  to  any  TSR,  say  (♦  ,{anp}}. 

CS 

Then: 
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(i)  E{X(f)}  =  E{  H  a  F.2{6(t-nT)))  (by  definition) 


■  p  n 


„  _  *  i 

=  E{  H  /  *(x+nT)**(x)dxF ,2{*  (t-nT)}}  (Eq.  (3-12)) 

_  n  P  l#  P 


p  n  0 
T. 


=  I  I  ■  /  E{x(T+nT)}$*(x)dT£’| ?(♦  (t-nT))  (assumed  valid) 

„  -  A  P  L  P 


p  n  0 


=  I  I  J  mx(T*nT)**(x)dxFL2{*  (t-nT))  (by  def.) 


p  n  0 


=  F  2{m  (t)}  (by  def.) 

*'p  * 

1  1 

,  T 


=  f  [  /  mx(x)«J(x)dx  l  ^(m/T)o(f-m/T):; 


P  0 


m 


Cii) 


E{X(f)X*(y)}  «  E{  l  l  a  a*  F . 2($  (t-nT) }F . 2{f* (s-roT) ) )  (by  Def.) 
pq  nra  ,  np  1  p  L  .q 


•  "ii  E<vVfL-vt-nT))v*;(=-*T)) 


pq  nm 


(assumed  valid) 


=  I  I  (k'mT) }  ^  def*  and  Eq>  (* 


pq  nm 


=  F*F  2{k  1  (t,s)}  (by  def.  and  Eq.  (3-2)) 

X  X  1 


*  K  (f,v) 

XX v  ’  ' 


(by  uef.) 


fl  '.^(vjym-Cv)  T^(f-v^T)  (Eq.  (3-4)). 
pq  1  ^  “  m 


OED 


It  should  be  mentioned  that  this  theorem  is,  in  a  sense,  the 
extension  of  Holly's  theorem  (Ref.  [>5,],  p.247) --us  applied  to  time 
averages  of  ergodic  stationary  processes-- to  ensemble  averages  cf  non- 
ergodic  cyclostationary  (and  stationary)  processes. 


-3): 


1 
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Note  that  the  expected  singular  behavior  of  Pu  {x}  is,  in  sore 

VC S 

than  one  May,  "expected"  since  it  shows  up  in  the  nean  function 

E{FU  (x(t)}},  as  a  string  of  inpulses,  and  in  the  autocorrelation 
CS 

function  E{FU  {x(t)}F*  {x(s)}},  as  a  string  of  inpulse  fences. 

HCS  "cs 


Note  also  that  since,  for  NSS  processes,  we  have 


V™  ■  lcxx'f>5'f-u> 

where  Kxx(f)  is  the  power  spectral  density  for  the  process  x,  then  we 
now  have  two  new  fornulas  for  the  power  spectral  density: 

Kxx(f)  «  /  E{X(f)X*(v)}dv  *  E{X(f)x(0)} 

•« 

where 


X(f)  »  {x(t) } . 

HCS 

The  first  topic  considered  in  the  next  section  provides  a 
concrete  example  of  the  utility  of  a  generalized  Fourier  transform  for 
CS  random  processes. 


3.  Haraonic  Series  Representation 


a)  Definition.  In  this  section  we  develop  a  particularly  interesting 
and  unique  translation  series  representation  which  is  intimately  related 
to  the  haraonic  TSR  discussed  in  Sec.  2d,  and  is,  in  fact,  the  frequency- 
time  dual  of  the  haraonic  TSR.  This  duality  is  inaediately  brought  to 
evidence  if  we  introduce  the  representation  in  the  following  aanner:  Ne 
begin  by  representing  the  Fourier  transfora  X(f)  of  the  T-CS  process  x(t) 
with  the  frequency -domain  harmonic  translation  series 


X(f)  "  E  ?  anp0n(f'p/T)’  (3-70) 

n  p  ” 

where  {©n)  is  the  CON  (in  L2[-1/2T,1/2T])  set  of  complex  exponentials 

0n(f)  «  Sf  e"j2imTfW(f)  (3-71) 


where  W  is  the  gate  function  defined  by 

1,  f  c  (-1/2T.1/2T] 
0,  ft  1-1/2T.1/2T] 


W(f) 


(3-72) 


and  where  the  representors 


}  are  given  by 


1/2T 

a  *  /  X(f+p/T)0* (f)df . 

np  _1/2T  n 


(3-73) 


Now,  if  we  take  the  inverse  Fourier  transform  of  this  frequency- 
domain  TSR,  then  we  obtain  the  following  TSR  in  the  time-domain: 


*ct) 3  i  i  aIlpyt-nT) 

n  p  r  r 


(3-74) 


where 


♦p(t)  «  /T  e^2lPt/T  w(t) 

and  w  is  the  inverse  Fourier  transform  of  the  gate  function  W 

wet)  -  sings'll  . 


(3-75) 


(3-76) 


Using  Parseval's  relation  (2]  in  Eq.  (3-73)  yields  the  time-domain 
formulas  for  the  representors: 

C» 

a  -  /  x(t*nT)$*(t)dt.  (3-77) 

r  r  «o»  V 

Thus,  wo  see  chut  our  frequency- time  dual  of  the  harmonic  TSR  of 
Section  2d  is  a  TSR  in  both  the  time-  and  frequency  domains. 

Now,  analogous  to  our  development  of  representations  in  terms  of 
PAM  processes  in  Section  2b,  we  collect  terms  in  our  time-domain  1SR  of 
Eq.  (3-74)  to  obtain 

x(t)  »  l  a  (t)  ej2npt/T  (3-78) 

n  * 


np 


(3-79) 


Employing  the  formulas  for  {a^}  in  the  above  equation  yields  the  more 
direct  formulas  for  the  continuous -time  representors: 


a  (t)  »  /  w(t-T)x(i)e'j2ni>T/Tdt. 

*  -  '13 


(3-80) 
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We  take  Eqs.  (3-78,80)  as  our  formal  definition  of  the 
harmonic  series  representation.21  Note  that,  in  teras  of  Eq.  (3-79),  we 
can  interpret  (a^t))  as  PAM  processes,  so  that  we  again  see  that  this 
heraonic  series  representation  (HSR)  is  the  frequency-tine  dual  of  the 
haraonic  TSR  of  Section  2d— this  tine  in  the  sense  that  the  continuous- 
tine  representors  in  the  haraonic  TJR  are  PAM  processes  with  pulses  which 
are  rectangular  (width  T)  in  the  tine  doaain,  and  the  representors  in  the 
HSR  are  PAM  processes  with  pulses  which  are  rectangular  (width  1/T)  in 
the  frequency  detain. 

For  this  haraonic  series  representation,  we  have  the  following 
fundaaental  theorem: 


THE0REM(3-7) :  Every  T-CS  process  x  adaits  the  mean-square  equivalent 
haraonic  series  representation 

g» 

x(t)  ■  l  a  (t)  e^2flpt^T  a.e.  t  e  (-*,») 

a  (t)  «  /  w(t-T)x( r)e"^“npT^Tdi 

I  .CD 

W(t)  «  (3, 


where  the  continuous-time  representors  tap)  are  jointly  WSS  and 
bandlimited  to  the  irt»  /al  £- 1/2T, 1/2TJ . 


2rthis  representation  has  previously  been  presented,  but  not  developed, 
by  Ogura  [6] ;  e.g>,  the  joint  wide-sense-stationarity  of  the  representors 
has  not  (to  my  knowledge)  been  recognized  except  in  our  preliminary 
paper  [43],  nor  has  the  fundamental  relationship  of  this  representation 
to  the  class  of  TSR’s  been  previously  discussed. 
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Proof :  Define 

e(t)  $  H{  | x (t)  -  l  a  (t)ej2,TPt/T|2}. 

P  p 

Expanding  the  square  and  interchanging  summation  and  expectation  yields 

e(t)  -  k  (t.t)  -  2Re[  l  E{a*(t)x(t)}e";|2npt/T] 

**  p  P 

♦  l  E{a*(t)a  (t)> 

p.q  P  q 

Substituting  the  given  expression  ror  ap  and  interchanging  summation 
and  integration  results  in 

CO 

e(t)  -  k  (t.t)  -  2Re[  /  J  w(t«T)kxx(t,T)e’j2,rp(t‘T)/TdT] 

-*  p 

P.q 

But,  from  the  Poisson  sum  formula  [2],  we  have 

l  w(t-t)e"j2,Tp(t“T)/T  »  T  l  w(t-x )6(t-T-qT) 

p  q 

=  6(t-r) . 


Hence, 

e(t)  =  kxx(t,t)  -  2Re[kxx(t,t)J*  kxxct,t) 

=0  . .e.  t  c  (-«,«) . 

Furthermore,  since  each  process  ap(t)  is  obtained  from  the  T-CS  process 
x(t)e”*'2l,pt^  via  a  bandlimiting  filter  (impulse  response  function 
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sin(irt/T)/irt)  with  passband  (-1/2T.1/2T] ,  then  Theorem  (2-3)  of 
Chapter  II  insures  that  ap(t)  is  a  WSS  process  which  is  bandlimited  to 
[-1/2T,1/2T] .  In  fact,  since  the  processes  {x(t)e*^2l,pt^T;  p  *  0,±1,±2,..} 
are  jointly  T-CS,  then  the  representors  {ap(i:);  p  ■  0,±1,±2,.~.) 
are  jointly  WSS. 

QED 

The  autocorrelation  function  for  every  T-CS  process  has  a  harmonic 
scries  representation  which  follows  directly  from  the  HSR  for  the 
process  itself: 


kxx(t,s)  *  l  kpq(t-s)ej2,,(Pt‘c>s)/T, 


p.q 


(3-82) 


where  k(t-s)  is  the  correlation  matrix  for  the  jointly  WSS  representors: 


.  II  w(t-1)w(s-Y)kjix(T,y)e"j2’,(pT',,1,)/TdTdY.  (3-83) 

■  CD 

Similarly,  the  double  Fourier  transform  of  the  autocorrelation  function 
has  the  representation 


K  ff.v)  »  l  K  (f-p/T)6(f-v(q-p)/T),  (3-84) 

p.q  pq 

where  K(f)  is  the  single  Fourier  transform  of  the  correction  matrix  k_C.t ) : 


K  (f)  =  F  [k  (t)j 
pq  pq 

1/21 

=  /  K  (f*p/T,v-q/T)dvW(f),  (3-8S) 

-1/2T  ** 


and  is  bandlimited  to  the  interval  (-1/2T.1/2T] . 
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It  is  interesting  to  note  that  the  correlation  matrix  K(f)  for 
the  HSR  of  any  T-CS  process  is  simply  related  to  the  correlation 
matrix  A(f)  corresponding  to  any  TSR  for  that  process  as  follows: 

Kpq(f)  ■  !Ta  ,V(f*p/T)V  ,.««♦;< (**<!/*>  (*-») 

where  {4>p}  are  the  Fourier  transforms  of  the  basis  functions  employed 
in  the  TSR.  Furthermore,  if  the  TSR  is  the  specific  one  from  which  the 
HSR  is  derived  (Eqs.  (3-74) -(3-77)),  then 

$p(f)  *  /f  W(f-p/T)  (3-87) 

and,  we  have 

Kpq(f)  "  W(f)Wf) 

CO 

Aoa(f)  *  l  KoQ^”n/T>*  (3-88) 

Thus,  the  HSR  correlation  matrix  K^(f )  is  just  a  band  limited 

(to  (-1/2T,1/2TJ)  version  of  the  l/T  periodic  TSR  correlation  matrix 

A(f);  i.e.,  K(f)  is  one  (centered)  period  of  A(f). 

Now,  substituting  Eqs.  (3-87,88)  into  Eq.  (3-7)  for  the  power 
spectral  density  of  the  stationarized  version  of  a  T-CS  process,  we  obtain 
the  HSR  formulas: 

00 

Kxx(f)  -  l  A  (f)W(f-p/T) 

AA  P». to  HP 

GO 

•  l  *  (f-p/T)  (3-89) 

ps-w  ‘ 

and  wc  see  that  the  stationary  character  of  a  CS  process  is  determined 
by  the  diagonal  elements  in  the  HSR  correlation  matrix. 
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b)  Resolution  and  reconstruction.  The  harmonic  series  re- ”  , mentation 
is  perhaps  most  transparent  when  viewed  in  terms  of  the  process-resolution 
Operation  which  is  defined  by  the  integral  equation  Eq.  (3-81)  of 
Theorem  (3-7).  Applying  the  convolution  theorem  to  this  equation  yields 
the  following  simple  formula  for  the  Fourier  transform  of  the 
representor  a^: 

A  (f)  *  X(f  ♦  p/T)W(f)  (3-90) 

r 

where  W  is  the  gate  function  defined  in  Eq.  (3-72).  Thus,  the 

resolution  operation  corresponds  to  equipartitioning  the  "frequency  line" 

into  support  intervals  of  length  1/T  upon  which  the  process  is  then 

decomposed  as  shown  in  Fig.  (3-10)  for  a  hypothetical  realization  of  the 

Fourier  transform  of  the  process. 

This  resolution  operation  has  the  straightforward  implementation 

shown  in  Fig.  (3-11)  where  W  is  the  ideal  low-pass  filter  with  transfer 

function  W(f).  Furthermore,  the  original  process  can  be  reconstructed 

from  the  representors  simply  by  shifting  them  (in  the  frequency  domain) 

and  adding  them.  The  corresponding  implementation  is  also  shown  in  Fig.  (3-10). 

Note  that,  paralleling  the  results  in  Section  2c,  a  resolution- 

reconstruction  system  which  decomposes  CS  processes  into  the  real  and 

imaginary  parts  of  the  representors  {a^(t)}  can  be  implemented  using  only 

P 

real  sinusoidal  multiplier  functions  in  the  structure  shown  in  Fig.  (3-11). 

c )  Properties  of  the  harmonic  series  representation.  In  this 
subsection,  we  present  several  properties  of  the  HSR  which  illustrate 
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its  convenience  for  characterizing  CS  processes. 

(1)  Stationarity:  Since  the  components  {ap(t)e,*2lT*>t^T}  of  the  HSR  are 
individually  WSS,  but  not  jointly  MSS  (because  of  the  exponential  factor), 
then  a  CS  process  will,  in  general,  be  WSS  if  and  only  if  the  various 
representors  {a^}  are  uncorrelated;  i.e.,  if  and  only  if  the  correlation 
matrix  is  diagonal. 

(2)  Phase  randomization:  If  x  is  the  process  which  is  derived  from  a 
T-CS  process  x  by  the  introduction  of  a  random  phase  variable  0: 

x(t)  «  x(t+6),then  as  discussed  in  Section  5c  of  Chapter  II  x  is  also 
T-CS.  Furthermore,  if  pQ (• )  is  the  PDF  for  0  and  Pg(*)  is  its  Fourier 
transform  (conjugate  characteristic  function),  then  the  elements  of  the 
correlation  matrix  X(f)  for  x  can  easily  be  shown  to  be  related  to  the 
elements  of  the  correlation  matrix  K(f)  for  x  by  the  formula: 

Wf)  ■  poup-<,,/T)iwf)‘  (3'91) 

Now,  since  pn(*)  is  a  PDF,  then  P„(r/T)  <  Pa(0)  =  1,  and  we  see  that  the 

O  0  a  o 

off-diagonal  elements  are  attenuated  by  the  phase-randomization. 

Of 

Furthermore,  if  P  ( (p-q)/T)  =  6  ,  then  h(f)  will  be  diagonal  and  x 

u  PH 

will  be  WSS  (a  result  previously  obtained  in  Theorem  (2-15)  of  Chapter  II). 

(3)  Bandlimitedness:  As  defined  in  Section  2a  of  Chapter  II,  a  random 
process  x  is  bandlimited  to  the  interval  [-B,B]  if  and  only  if  the  double 
Fourier  transform  of  its  autocorrelation  function  satisfies  the 
bandlimiting  constraint: 

Kxx(f,v)  -  0 

for  jf|  2  B  and  for  |v|  >  B.  N^w.  using  Hq.  (5-85)  and  this  definition, 


173 


it  is  easily  shown  that  if  x  is  a  T-CS  process  which  is  bandlimited  to 
l-(M+l/2)/T,(M+l/2)/T]  then  x  admits  an  Mth  order  HSR;  i.e.,  all 
representors  with  indices  greater  (in  magnitude)  than  M  are  zero,  and 
the  correlation  matrix  is  (2M*1)  dimensional.  Similarly,  if  x  is  a 
"band-pass"  process: 

■  0 

for  | | F | - f  |  >  B,  and  for  jjv|-fQj  >  B,  then  x  admits  an  HSR  with  only 
2M  terms  (where  B  =  (M*l/2)/T). 

(4)  Time- invariant  filtering:  If  x  is  the  output  of  a  time- invariant 
filter  with  T-CS  input  x,  then  as  discussed  in  Sec.  2a  of  Chapter  II,  x 
is  also  T-CS.  furthermore,  if  (!  is  the  transfer  function  for  the  filter, 
then  the  elements  of  the  correlation  matrix  jfc(f)  for  x  can  be  related  to 
the  elements  of  the  correlation  matrix  !K(f)  for  x  by  the  formula: 

Kiq(f)  =  W ( f )G ( f ♦p/T )G*  ( f +q/T) K  ( f ) .  (3-92) 

Note  that  if  G  is  an  ideal  low -pass  filter  with  cutoff  frequency  fQ,  then 

ah'  elements  in  the  matrix  K(t')  i  iti,  either  index  greater  (in  magnitude) 

A 

than  M  -  min{n;  n  >  T f Q 1  / 2 }  will  be  zero,  so  that  x  can  be  represented 
by  an  M**'  order  HSR--a  result  which  is  ubvious  from  t!ie  above  discussion 
on  bandlimiting. 

d)  Examples.  In  this  subsection,  we  present  several  specific  types 
of  CS  processes  which  admit  particularly  simple  finite  order  USR's. 
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(1)  Amplitude-modulated  signals:  Our  first  example  of  a  CS  process 
introduced  in  Chapter  II  (Sec.  2b)  was  the  amplitude-modulated  (AM) 
signal  with  realizations  of  the  form: 

x(t)  *  p(t)y(t) 


where  p  is  a  T-periocuc  deterministic  function  and  y  is  a  MSS  process. 
Now  if  p  has  the  finite  harmonic  Fourier  series  representation 

pet) .  I 

q»-M  q 

and  y  is  bandlimited  to  (-1/2T,1/2T]  then  x  admits  an  M  order  HSR  with 
representors 


a  y(t),  p  e  0,±1,±2, . . . . ,±M 
a  (t)  »  p 

P  0,  jp|  >  M. 

and  if  p  is  a  sinusoid  (as  it  most  often  is  in  communications  applications), 
then  M  *  1,  and  the  correlation  matrix  has  the  form: 


K(f )  *  K  (f) 

yy 


1  0  1 
0  0  0 
1  0  1 


(2)  Frequency-division-raultiplexed  signals:  It  was  shown  in  Section 
2b  of  Chapter  II  that  a  multiplicity  of  T-CS  AM  signals  can  be  added  to 
form  a  composite  T-CS  frequency-division-multiplexed  (FDM)  'signal  with 
realizations  of  the  form 
M 

x(t)  =  £  y  (t)cos(u)Qt  ♦  2:/pt/T) 

p=l  p 


where  u>o  is  an  integer  multiple  of  2ir/T,  and  {y^}  are  each  banllimited 
to  [-1/2T.1/2T] .  Now,  by  expanding  the  sinusoid  into  the  sum  of 
exponentials,  we  obtain  the  HSR 


x(t) 


(M+N) 

■  l 

p—  (M*h) 


a  (t)ei2*P‘/T 
P 


9 


where  N  *  woT/2ir,  and 

}yp(t),  p  -  ±(N+l),±(N+2) , . .  .±(N*M) 
0,  N+M  <  |p)  <  N. 


ap(t) 


If  the  components {y^}  are  uncorrelated,  then  the  composite  FDM  signal 
has  a  sparse  correlation  matrix  of  the  form: 


K  * 


K(-N-M)(-N-M) 

0 

'  K(-N)(-N) 

0  0 

K(N)(-N) 

0 


K(-N-M) (N+M) 

K(-N)(N) 

0 

K(N)(N) 


K(N+M)(-N-M) 


K(N+M) (N+M) 


with  elements  given  by 


(f) 


\  Kr(f),  |p|  =  |qj  «  N  ♦  r 
0,  otherwise 


where  K^ff)  is  the 


power  spectral  density  for  y  . 
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(3)  Random  periodic  signals:  In  Section  Sa  of  Chapter  II,  we  discussed 
the  random  periodic  signal  as  a  reasonable  model  for  many  biomedical  and 
communications  test  signals  (modeled  at  the  receiver).  These  signals 
can  be  expressed  in  terms  of  a  Fourier  series 

x(t)  .  I 

p*-M  P 

where  (a  }  are  random  Fourier  coefficients.  Clearly  then,  these  signals 
P 

admit  M**1  order  HSR’s  with  representors  a^(t)  «  a^,  so  that  the 
correlation  matrix  K(f)  factors  into  the  product  of  a  constant  matrix 
and  an  impulse  function. 

e)  Solution  of  integral  equations  and  realization  of  periodically 
time-varying  systems.  As  discussed  in  Section  2f,  the  TSR  for  an 
autocorrelation  function  can  be  employed  to  convert  an  integral  equation 
to  a  matrix  equation  which  can  be  solved  by  matrix  inversion.  Now,  since 
the  HSR  is  derived  from  a  TSR,  then  it  too  can  be  similarly  employed.  In 
fact,  using  Eqs.  (3-75),  (3-88)  in  Theorem  (3-2)  yields  the  solution 

h(t,T)  =  l  h  (t-T)ej27T(pt"qT)/T  V- t.r  e  (-<»,»)  (3-93) 

p*q  pq 

to  the  integral  equation 

CO 

/  k(t,s)h(s,T)ds  +  Ah(t,T)  =  g (t , t)  V-  t,r  c  (-»,»)  (3-94) 

•  00 

where  the  (h  }  are  the  inverse  Fourier  transforms  of  the  elements 

pq 

of  the  matrix 
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H(f)  = 


[K(f)  ♦AIj-1  G(f), 


Y  f  e  [-1/2T,1/2T] 

Y  f  t  [-1/2T.1/2T3 


(3-93) 


where  K,  G  are  the  Fourier  transforms  of  the  bandlimited  (to  [-I/2T.1/2T}) 

I 

matrices  k,  £  which  represent  the  T-periodic  kernels 

*  '  .  \ 

i 

1  k(t.'s)  =  l  k 
P.q  W 


g(t,s) 


(t-s)ej2l,(pt'qs)/T 


(3-96) 


as  in  Eqs.  (o-83,  85).  ,  .  , 

! 

.  Furthermore,  if  this  solution  is  to' be  implemented  as  the  impulse 
response  of  a  linear  system- -as  discussed  in  Section  2g--thon  the 
straightforward  implementation  shown  in  Fig.  (3-12)  suffices.  Note 
t;hat  this  linear  system  is  composed  of  a  matrix  of  time- invariant  filters 
(with  transfer  functions  H  (f))  which  is  flanked  at  the  input  and 

i  pq  r 

output,  respectively,  by  the  resolutidn  and  reconstruction  systems  of 
rig.  (3-11).  Note  also  that  since  II  (f)  is  band  limited,  then  it  can  be 
exactly  realized  with  a  matrix  sampled-data  filter,  with  transfer  function 


[53,54] 


B(f)  =  l  11(4: -p/T) , 

p=-“ 


flanked  at  the  input  and  output,  respectively,  by  T-periodic  impulse- 
samplers  and  ideal  low-pass  filters  (W(f)). 

Finally,  it  should  be  mentioned  that  such  a  system  is  not  causal 
and  cannot  be  physically  implemented  without  the  incorporation  of  a  uel-ty 
as  discussed  in  Section 


i 
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As  an  exaaple  of  this  integral -equation  solution  method,  consider 
the  special  case  (which  occurs  frequently  in  practice)  where  the  kernels 
g  and  k  in  Eq.  (3-94)  are  equal.  Now  if  k  is  the  autocorrelation  function 
for  the  FDM  signal  discussed  in  the  previous  subsection  (or  has  the  sane 
fora),  then  the  identical  matrices  G,  K  have  the  sparse  fora: 

jKr(f),  jpj  *  |q|  =  N  ♦  r 

0,  otherwise 

sc  that  the  solution  matrix  of  Eq.  (3-  95)  has  eleaents  given  by  the 
formula 

\  4-K(f) 

Ip!  *■  |q|  -  N  ♦  r 


<Wf)  *  Wf) s 


H  (f)  = 
pq 


1  «r(f> 

\  Kr(f)  ♦  A  ’ 
0, 


(3-97) 


otherwise. 


In  the  next  chapter  we  will  extend  the  integral -equation  solution 
method  of  Eqs.  (3-93)- (3-96)  to  include  the  case  where  the  kernels 
k,  g  are  not  only  jointly  T-periodic  in  both  variables,  but  also 
individually  T-periodic  in  each  variable  (as  is  the  case  for  the 
autocorrelation  function  of  a  random  T-periodic  signal),  and  where  the 
interval  of  integration  is  finite. 


4.  Fourier  Series  Representation  for  Autocorrelation  Functions 


a)  Definition.  In  this  section  we  briefly  consider  the  harmonic 
Fourier  series  representation  for  autocorrelation  functions  for 
cyclostationary  processes.  In  contrast  to  the  autocorrelation  function- 
representations  presented  in  the  previous  two  sections,  this  representation 
does  not  result  from  a  representation  for  cyclostationary  processes : 
there  is,  in  fact,  no  process-representation  from  which  the  Fourier  series 
representation  for  an  autocorrelation  function  automatically  results.?2 

If  x  is  a  cyclostationary  process  with  period  T,  then  its 
autocorrelation  function  is— by  definition—joiritly  T-periodic  in  its 
two  arguments: 

kxx(t+T,s+T)  =  hxx(t,s)  V-t,s. 

Now,  if  we  define  a  new  function  of  two  variables  by  making  the  change 
of  variables: 

r  (t',s')-k  (s'+t72,s'-t'/2),  j 

XX  v  XX 

then  we  have 

kxx(t,s)  =  rxX(t-s,(t^s)/2)  (3-99) 

so  that  the  first  argument  in  r  is  the  time-difference  or  size  (and 
sense)  of  the  time-interval  (t,s)  and  the  second  argument  is  the  time- 
average  or  midpoint  of  the  time-interval  (t,s).  Furthermore,  the  section 


22 This  is  a  conjecture  which  appears  to  be  difficult  to  prove. 
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i 


function  r  (t1,*)  is  T-periodic  for  every  t',  and  the  section  function 

AA 

i  , 

r  (-,»')  is  a  "decaying"  function  and  will  be  contained  in  L2(t“,°p) 
for  most  processes  of  interest;  i.e.,  those  processes,  whose  correlation 
goes  to  zero  faster  than  l/|t-s|. 

We  now  represent  the  T-periodic  functions  (rxx(t' ,  ^ ;  t'  c  (-«>,®)}! 
with  their  harmonic  Fourier  series: 


r  (t\s’)  =  l  c  (t')e;i2lins,/T 
i  xx  '  L  n  ,  J 


n=-»  i 


where 


T/2 


eft')  =  i  /  r(t',s')e'j2^s,/Tds' 


T_t^2  xx 


(3-100) 


If  we  substitute  Eq.  (3-100)  into  Eq.  (3-99),  then'  we  obtain 


kxx(t,s)  =  l  cn(t-s)e 


j  Trn  (t.+s  ) /T 


n=-» 


T/2 


c  (t)  =  ~  /  k  (s+t/2,s-t/2)e'j2Trns/Tds 

II  *  2  , 


(3-101) 


i  \ 

which  is  the  Fourier  series  representation  (FSR)  ,for  'the  autocorrelation 
function  . 23  , 

t  I 

Corresponding  to  this  FSR  for  the  autoebrreiation  function  k  ,  w;e 

XX 

have  the  following  representation1  for  the  double  Fourier  transform  ofKxx:. 


23  This  representation  has  been  considered  in  great  depth  by  Hurd  [5].  In 
the  third  Chapter  (which  is  devoted  entirely  to  this  representation)  of 
h:s  doctoral  dissertation  on  cyclostationary  processes,  he  investigates 
various  modes  of  convergence  of  the  partial  sums  of  this  Fourier  series; 
However,  neither  Hurd,  no^  any  of  the  previous  investigators  referenced 
by  Hurd,  appear  to  have  identified  the  direct  relationship  existing  between 
the  Fourier-coefficient 'functions  and  the  process  as  described  in  the 
remainder,  of  this  subsection. 


1S1 


Kxx(f,v)  =  l  Cr (f-n/ 2T) 6 (v-f +n/T)  (3-102) 

n 

where  C  is  the  single  Fourier  transform  of  c  .  Notice  that  if  we 
n  n 

substitute  this  FSR  into  Eq.  (3-85),  then  we  obtain  the  following 
relationship  between  the  elements  of  the  correlation  matrix  for  the 
HSR  of  Section  3,  and  the  coefficient-functions  for  the  FSR: 

Kpq(f)  =  Cp-q(f+(P+q)/2T)W(f)*  (3‘103) 

Furthermore,  by  equating  the  right  members  of  Eqs.  (3-102),  (3-84),  we 
obtain  the  relation: 

Cn(f-n/2T)  =  l  K(pn)p(f-p/T),  (3-104) 

so  that  the  n^  FSR  coefficient  function  is  equal  to  the  sum  of  the 
frequency-translated  versions  of  the  elements  of  the  n  n  off-diagonal 
in  the  HSR  correlation  matrix  K(f). 

In  view  of  the  fact  that  the  HSR  and  the  FSR  for  the  autocorrelation 
functions  of  CS  processes  both  employ  the  harmonic  exponentials  as  basis 
functions,  one  might  expect  to  find  some  interesting  relationships 
between  the  two  representations--in  addition  to  Eqs.  (3-103),  (3-104). 

The  following  is  perhaps  the  most  fundamental  relationship  existing 
between  the  HSR  and  the  FSR: 

From  Section  3,  we  know  that  the  elements  of  the  correlation  matrix 
for  the  HSR  are--by  definition--cross-correlation  functions  of  jointly 
WSS  complex  processes: 

k  (t-s)  =  E{a  (t)a* (s) } 
pq  p  q 
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where  the  process  ap(t)  is  obtained  by  reducing  the  T-CS  process 
x(t)e  to  a  wss  process  via  bandlimiting  to  the  frequency- interval 

[-1/2T,1/2TJ  (Theorem  (3-7)).  Similarly,  the  FSR  coefficient-functions 
are  crosscorrelation  functions  of  WSS  complex  processes: 

cp(t-s)  =  E{bp(t)bp(s)> 

where  the  process  bp(t)  is  obtained  by  reducing  the  T-CS  process 
x(t)e  to  a  WSS  process  via  phase-randomization. 

We  state  this  new  result  as  a  theorem: 


Now,  the  coefficient  functions  in  the  FSR  are  equal  to  the  cross¬ 
correlation  functions  of  the  {b  }: 

P 

cp(t-s)  =  E{bp(t)bp(s)}.  (3-106) 

Proof: 

E{bp(t)bp(s)}  =  E{xp(t+0)xp(s+e)} 

00 

=  /  E{x  (t+o)x  (s+o)}P  (o)da 

_  00  P  P 

T/2 

=  h  /  E{x(t+o)x(s+a)}e  ^7IP^t+s+2a''^r»n 
1  -T/2 

.  i  SVl  (t,o,s*<,)e-j*P(t*s)/Te-52'Po/Tda 

1  T?  /  *■>  XX 

=  1  ?'\  c  (t.s)ej"(q-P)(t^)/T  e  j2"(q-p)o/Tdo 

1  -T/2  q  q 

=  l  c  (t-s)ejn(q-p)(t+s)/Ti  JT/2ej2ll(q-P)a/Tdo 
q  q  T  -T/2 

=  c  (t-s). 

P 

QED 


Note  that  since  bp  is  a  complex  process  (for  p  /  0) ,  then  cp(t-s) 

is  not  the  autocorrelation  function  for  b  ,  but  rather  the  crosscorrelation 

P 

function  for  b  and  b*.  Note  also  that  since  x  is  real,  then  b*  =  b 

P  P  P  -P 

The  system  which  transforms  any  T-CS  process  x  into  the  WSS  processes 


(bp)  is  shown  in  Fig.  (3-13)  where  e  is  a  delay  line  whose  delay  length 
is  a  random  variable  uniformly  distributed  over  the  interval  [-T/2,T/2]. 


Hence,  unlike  the  resolution  systems  for  the  TSR's  and  the  HSR,  this 
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transformation  is  a  random  periodically  time-varying  linear  system 
(as  discussed  in  Section  5  of  Chapter  II),  and  has  random  impulse- 
response  function 

gp(t,r)  =  e"^  1tPt/Ts  (t+0-x) . 

Observe  that  this  random  system  has  the  same  form  as  the  HSR  resolution 
system  of  Fig.  (3-11)  except  that  "ph^se -randomizers"  are  used  in  place 
of  "bandlimiters". 

The  p  inverse  transformation  which  converts  b  back  into  the 
^  P 

original  process  x  is  also  a  random  periodically  time-varying  linear  system, 
and  is  statistically  dependent  on  the  resolution  transformation  as  well 
as  on  its  input  bp.  The  random  impulse-response  function  for  this  inverse 
transformation  is 

gZ 1  (t ,  t)  =  eJ1Tpt/T6(t-0-T), 


so  that  the  output  process  is 


x  (t)  =  bp(t-6)ejTrpt/T 


where  0  is  the  same  random-phase  variable  employed  (in  gp)  to  resolve 
x  into  b  . 


b)  Properties  of  the  Fourier  series  representation.  In  this 
subsection,  we  present  various  properties  of  the  FSR  which  are  useful  in 
analyses  dealing  with  CS  processes: 

i)  Phase  randomization  and  stationarity.  If  the  process  x  is  obtained 
from  the  T-CS  process  x  by  the  introduction  of  a  random-phase  variable  0: 


-  — %zr> 


185 


x(t)  =  x(t+0) 


then  the  FSR  coefficient-functions  {c  }  for  the  autocorrelation  function 

n 

a# 

for  x  are  given  by 


cn(t)  =  P0(n/T)cn(t) 


(3-107) 


where  (c  }  are  the  FSR  coefficient-functions  for  x,  and  P.  (•)  is  the 
n  0 

Fourier  transform  of  the  PDF  for  6,  p  ( * ) -  Now,  if  the  PDF  is  uniform,  then 

0 


P0  (n/T)  = 


1,  n  =  0 
0,  n  +  0 


so  that 

and 

=  So  (t_s)  * 

xx 

Hence,  the  zeroth  coefficient  in  the  FSR  is  the  autocorrelation  function 
for  the  IVSS  process  obtained  via  phase-randomization.  This  result  is 
more  directly  obtained  by  noting  that 


0, 

c  (t)  = 
n 


c0(t),  n  =  0 


0, 


n  /  0 


c0(t-s)  =  E{bo(t)bQ(s)} 

=  E{x(t+O)x(s+0) } 


(3-108) 


where  0  is  the  uniformly  distributed  random-phase  variable. 

ii)  Filtering  and  bandlimiting .  If  x  is  the  output  of  a  linear  time- 

invariant  filter  with  transfer  function  G(f),  and  with  T-CS  input  x,  then 

'V 

the  Fourier  transform  of  the  FSR  coefficient-functions  for  x  are  given  by: 
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£n(f)  =  G(f+n/2T)G*(f-n/2T)Cn(f)  (3-109) 

where  {C  }  are  the  Fourier  transforms  of  the  FSR  coefficient -functions 
n 

for  x.  Now,  if  G  is  a  bandlimiting  filter  with  bandwidth  2B: 

G(f)  =  0,  | f |  >  B, 

then 

t  (f)  =  0,  n  >  2BT, 
n  = 

so  that  bandlimited  processes  possess  only  a  finite  number  of  non-zero 
coefficient-functions.  This  is,  of  course,  true  for  "band-pass”  as  well 
as  "low-pass"  processes.  Note  that  if  B  <  1/2T,  then  only  the  zeroth 
coefficient-function  is  non-zero  and  the  bandlimited  process  is  WSS--a 
result  previously  obtained. 

iii)  Constraints  on  coefficient-functions.  There  are  a  number  of 
interesting  properties  possessed  by  the  FSR  coefficient-fur^  .'.ions . 

These  properties  can  be  viewed  as  constraints  which  must  be  satisfied  by 
the  coefficient-functions  corresponding  to  any  CS  process.  We  briefly 
discuss  three  particularly  interesting  types  of  constraints: 

(1)  Symmetry  constraints:  Using  the  defining  equation  (Eq.  (3-102)), 
for  the  FSR,  one  can  easily  show  that  the  coefficient-functions  {c  } 
and  their  Fourier  transforms  (Cn)  exhibit  the  symmetries: 

c  (t)  =  c*  (-t)  =  c  (-t) 
n  -n  n 

C  (f)  =  C*  (f)  =  C  C-f)  (3-110) 

n  -n  n 


where  the  last  equality  in  each  line  is  valid  only  if  x  is  a  real  process. 
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(2)  Norm  constraints:  Employing  the  Cauchy-Schwarz[2]  inequality  for 
inner  products  (in  H^)  results  in 

kxxCt’s)  :  kxxtt,t)kxx(s's)- 

Now,  using  the  FSR  for  kxx  and  replacing  t  by  (o+t/2),  and  s  by  (o-t/2) , 
and  integrating  w.r.t.  a  over  the  interval  [-T/'2,T/2]  yields  the  bound 

I  |cn(t)!2  <  I  lcn CO) | 2  e32TOt/T  <  l  |c  (0)|2.  (3-111) 

n  n  n 

Furthermore,  if  we  again  integrate  (over  [-NT/2,NT/2] ) ,  then  we  obtain 

the  bound 

.  NT/ 2 

l  I  |c(t)|2dt  <  |c  (0)|2.  (3-112) 

n  N1  -NT/2  n  "  U 

(5)  Support  constraints:  Since  the  mean-squared  value  of  any  random 
variable  must  be  non-negative,  c..en  by  considering  the  particular  random 
variable 

00 

y  =  /  g(t)x(t)dt, 

.00 

where  x  is  a  T-CS  process,  and  g(*)  is  any  real  function,  we  obtain  the 
non-negativity  condition 

00 

E{y2}  =  //  g(t)g(T)k  (t.T)dtdi  >  0  (3-113) 

.00 

Now,  using  Parseval's  relation  [2],  we  obtain 

CO 

E { y 2 }  =  //  G (f )G* (v)K  ,(f,v)dfdv  >  0  (3-114) 

XX  “ 

.co 

and,  using  the  FSR  of  Eq.  (3-113)  for  K  yields 

CO 

'  /  G(t>n/2T)G*(f-n/2T)C  (fldf  2  0  . 

n  ->« 


(3-115) 
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This  general  non-negativity  condition  imposes  various  interesting 
constraints  on  the  support  of  the  Fourier  transforms  of  the  FSR 
coefficient-functions.  Specifically,  if  we  choose  a  "double  band-pass" 
shape  for  G: 


G  (f)  = 


f  A,  f  e  [m/2T-fo-B,m/2T-fo+B]U[m/2T+fo-B,m/2T+fo+B] 


A*,  f  e  [-m/2T-fo-B,-m/2T-fo+B]U[-m/2T+fo-B,-m/2T+fo+B] 


0,  otherwise 


where 


0  <  f  +  B  <  1/4T, 
=  o  = 


then,  Eq.  (3-124)  becomes 

-f  +B  f  +B  -f  +B 

Re{A2  f°  C  (f)df  +  A2  /  0  C(f)df)+|A|2  /°  Cn(f+m/2T)df 

-f  -B  m  f  -B  m  -f  -B 

0  0  o 

f  +B 

+  |A | 2  /  0  Cn(f+m/2T)df  >  0  . 

f  -B  U 
o 

Now,  by  choosing  A  equal  to  1  and  to  j ,  we  obtain  the  constraint 

B  B 

/  [C0(f+m/2T+fo)+C0(f+m/2T-fo)]df  >  |  /  Re{Cm(f+fo)+Cm(f-fo) }df | 
-B  -B 


where, 


0  <•  f  +  B  <  1/4T. 
=  o  = 


(3-116) 


Among  other  things,  this  is  a  constraint  on  the  support  of  the  FSR 
coefficient-functions.  From  it,  we  conclude  that  if  the  m**1  coefficient- 
function  has  support  throughout  the  frequency  band  [-1/4T,1/4T] ,  then 
the  zeroth  coefficienfrfunction  must  have  support  throughout  the 
frequency  band  [m/2T-l/4T,m/2T+l/4T] .  Other  constraints  can  be  derived 
by  choosing  other  types  of  gate-functions  for  G.  In  fact,  it  appears  as 
though  the  jn  coefficient-function  Cm  will  have  support  throughout  the 
frequency  band  [-B,B]  for  any  B  if  and  only  if  the  zeroth  coefficient- 
function  has  support  throughout  the  band  [m/2T-B,m/2T+B] . 

c)  Solution  of  integral  equations,  and  realization  of  periodically 
time-varying  systems.  In  Subsections  2f,  3e,  we  saw  that  series 
representations  for  autocorrelation  functions  (and  other  periodic  functions 
of  two  variables)  can  be  employed  for  solving  linear  integral  equations 
in  which  these  functions  appear.  This  was  possible  for  the  TSR's  because 
of  the  separation  of  variables  effected  by  the  representation.  Although 
the  FSR  does  not  separate  variables,  it  does  provide  a  means  for  converting 
the  integral  equation  to  a  set  of  linear  algebraic  equations.  However,  as 
we  will  see,  finite  order  FSR's  do  not  convert  integral  equations  to  a 
finite  set  of  algebraic  equations  as  was  the  case  for  TSR's.  The  set  of 
algebraic  equations  obtained  with  the  FSR  is  infinite,  and  not 

particularly  easy  to  solve  except  in  special  cases. 

Using  Parseval's  relation  [2],  the  linear  integral  equation 
00 

/  h(t,s)k(s,T)ds  +  Ah(t,x)  =  g(t,x)  V-  t,x  e  (-00,00)  (3-117) 

-  CO 

can  be  converted  to  the  equivalent  (dual)  linear  integral  equation 
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/  H(f,  v)K(v,u>)dv  +  XH(f,(ti)  =  G(f,aj) ,  V-  f,<i>  e  (-»,»)  (3-118) 

where  K,  H,  G  are  the  double  Fourier  transforms  Of  k,  h,  g.  Now,  if 
k,g  are  jointly  T-periodic  in  their  two  variables,  as  are  the  auto¬ 
correlation  and  crosscorrelation  functions  for  T-CS  processes,  then  they 
admit  FSR’s,  and  their  double  Fourier  transforms  can  be  expressed  in  the 
form 

K(f,v)  =  l  Cn(f-n/2T)6(v-f+n/T) 
n 

G(f , v)  =  l  G  (f-n/2T)6(v-f+n/T).  (3-119) 

n 

Assuming  the  same  form 

H(f , v)  ■  l  H  (f-n/2T) 6 (v-f+n/T)  (3-120) 

n 

for  the  solution  function,  and  substituting  into  the  integral  equation 
yields,  after  integration,  the  following  equivalent  set  of  linear 
algebraic  equations: 

00 

l  C  (f-(n+m)/2T)H  (f-m/2T)  +  AH  (f-n/2T) 
m=-°° 

=  G  (f-n/2T)  V-n.  (3-121) 

There  does  not  appear  to  be  any  way  to  obtain  a  general  closed 
form  solution  to  this  infinite  set  of  equations; 21*however,  if  {G^} 

satisfy  certain  banalimiting  constraints,  then  solutions  are  possible.  We 
consider  the  special  (but  common)  case  where  g  =  k.  Now,  if  the  {C^} 
satisfy  the  constraint: 

24  This  difficulty  appears  to  be  related  to  the  fact  (conjecture)  that  the 

FSF  for  autocorrelation  functions  does  not  result  from  a  process-representation 
as  do  the  HSR  and  TSR's. 


191 


then  the  solutions  to  Eq.  (3-121)  become: 

C  (f-n/2T) [1 -H  (f)] 

H  (f-n/2T)  =  — - 2 -  ,  n  *  0,±1 , . . . ,± (M-l) 

A  +  C0(f-n/T) 

and  {H^;  n  =  0,±1 , . . . ,±(M-1) }  can  be  obtained  by  inverting  a  (2M-l)x(2M-l) 
matrix  of  functions. 

As  a  simple  example  of  this  integral-equation  solution  method, 
consider  the  autocorrelation  function  corresponding  to  an  AM  process 
(see  Sec.  2  of  Chapter  II)  as  a  kernel: 
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Hence;  the  solution  to  the  original  integral  equation  is: 

h(t,x)  =  cos(2irt/T)cos(2TTT/T)r(t-x) , 

where  r (• )  is  the  inverse  Fourier  transform  of  R(*)* 

If  the  solution  to  our  integral  equation  is  to  be  implemented  as 
the  impulse  response  of  a  linear  system--as  discussed  in  Subsections 
2g,  3e--then  the  straightforward  FSR  realization  shown  in  Fig.  (3-14a), 
can  be  employed.  The  solution  to  the  integral  equation  will  not,  in 
general,  be  the  impulse-response  function  of  a  causal  system,  so  that 
the  time- invariant  path-filters  shown  in  Fig.  (3-14a)  will  not  be  causal. 
However,  if  one  is  given  a  causal  impulse-response  iunction  h  to  start 
with,  then  the  coefficient-functions  {h^}  in  its  FSR, 

T/2 

h  (t)  =  i  /  h(s+t/2,s-t/2)e"^2ims/Tds,  (3-124) 

n  _T/2 

=  0  for  t  <  0. 

will  be  the  impulse-response  functions  of  causal  time-invariant  filters, 

so  that  the  realization  shown  in  Fig.  (3-14a)  will  then  be  causal. 

This  FSR  realization  can  be  simplified  for  the  purpose  of 

implementation.  Such  a  simplified  version  is  shown  in  Fig.  (3- 14b) 

where  only  input  modulators  are  used,  and  all  devices  are  real.  The 

inpulse-response  functions  for  the  path-filters  with  transfer  functions 

G  , ,  G  „  are  related  to  the  original  FSR  coefficient-functions  as  r  'llows: 
nl  n2  b 
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i 

gnl(t)  =  2Re{hn(tj}cos7rnt/T  -  2Im{hn(t)  }sinmt/T 


gn2(t)  =  _2^e^n(t) }sinnnt/T  -f2Im{h  (t)}cosirnt/T  . 


I 


S 


(3-125) 


! 


Figure (3-1)  Multiplier- integrator  realization  of  the  piocess- 


resclution  device. 


X(t) 


h(t) 


yt  — 


h(t)  = 


Figure(3-2)  Filter-sampler  realization  of  the  process-resolution 


device. 


^jpv-<o  ■<^.A^rt  '-r.'--.~'~  vt“  ‘ 
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representor  a  (t)  and  a  discrete  representor{ 
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Figure(3-13)  Transformation  of  n’-CS  nrocess  x  into  WSS  complex 
processes  {b^}  whose  crosscorrelations  are  the  FSR  coefficient- 
functions  {Cpl-  (The  random  delay  0  is  uniformly  dis tributed  over  fO,T].) 


sin  (2nMt/T) 


Figure(3-14b)  Simplified  realization  of  a  periodical ly(T)  time-varying 
linear  system  whose  impulse-response  function  has  an  Mu  order  FSR. 
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CHAPTER  IV 

LEAST-MEAN-SQUARED-ERROR  LINEAR  ESTIMATION 
OF  CYCLOSTATIONARY  PROCESSES 

1 .  Introduction 

The  material  in  the  preceding  two  chapters  serves  mainly  as  a 
theoretical  base  upon  which  various  problems  dealing  with  random 
periodic  phenomena  can  be  formulated  and  solved.  Chapter  II  provides  a 
base  for  obtaining  mathematical  models  for  physical  systems  and  signals , 
so  that  physical  problems  can  be  formulated  mathematically,  and 
Chapter  III  provides  a  base  for  simplifying  the  mathematical  models  so 
that  analyses  can  be  simplified,  and  solutions  obtained. 

In  this  fourth  chapter,  we  consider  a  particular  application  of  the 
theory  developed  in  these  earlier  chapters.  Although  this  application 
is  but  one  of  many  possibilities,  it  has  special  significance  in  that  it 
has  been  the  primary  motivation  for  developing  the  theory.  Furthermore, 
the  problem  considered  in  this  application  is  of  significant  practical 
(as  well  as  theoretical)  value  in  that  its  solution  provides  a  means  for 
improving  the  quality  of  present  communication  systems,  and  a  means 
for  evaluating  certain  performance  measures  for  newly  proposed  (as  well 
as  existing)  communication  systems,  and  it  leads  to  useful  insight  into 
the  nature  of  optimum  signal  processing. 

Specifically,  the  problem  to  be  considered  is  that  of  least-mean- 
sqitared-error  linear  estimation  of  imperfectly  observed  cyclostationary 
processes--or ,  in  communications  jargon,  optimum  filtering  of  noisy 
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distorted  cyclostationary  signals.  The  general  form  of  this  optimum 
filtering  problem  is  shown  pictorially  in  Fig.  (4-1):  A  random  signal 
process  x  is  transmitted  through  (and  distorted  by)  a  channel  vhich  is 
composed  of  a  dispersive  transformation  which  can  be  nonlinear,  time- 
varying,  or  even  random;  and  noise  which  can  be  additive  and/or  multi¬ 
plicative.  The  received  signal  y  (the  distorted  version  of  x)  is 

processed  by  an  "optimum  filter"  which  is  designed  to  minimize  the  mean- 

* 

squared  difference  between  its  output  x(t)  and  the  transmitted  signal 
x(t)  for  all  values  of  t  in  soae  interval  Tq  (usually  (-“,“)). 

Typically,  the  filter  is  chosen  to  be  optimum  subject  to  the 
constraint  that  it  be  linear.  Nonlinear  processing  of  the  received 
signal  is  usually  carried  out  in  the  receiver  which  follows  the  optimum 
filter.  Such  nonlinear  signal  processing  will  not  be  considered  here. 

It  should  be  mentioned,  however,  that  if  the  transmitted  signal  x  is  a 
Gaussian  process,  and  if  the  channel  is  composed  of  a  linear  deterministic 
transformation  and  additive  Gaussian  noise,  then  the  optimum  (least-mean- 
squared  error)  filter  subject  to  no  constraints  will  be  a  linear  filter  [51]. 

The  particular  aspects  of  the  optimum-filtering  problem  that  will 
be  investigated  in  this  chapter  are  the  structure  and  the  performance  of 
optimum  (linear)  filters  for  cyclostationary  signals.  In  fact,  we  will 
prove  (and  demonstrate  with  numerous  examples)  the  proposition  that 
optimum  filters  for  cyclostationary  signals  are  periodically  time-varying 
systems  which  can  yield  significantly  improved  performance  over  optimum 
filters  which  are  constrained  to  be  time-invariant  and  which  thereby 


We  will 


ignore  the  cyclic  character  of  the  cyclostationary  signals.25 
also  prove  (and  demonstrate  with  examples)  the  fact  that  optimum  linear 
receivers  consist  of  demultiplexor-demodulators  followed  by  time- 
invariant  filters— the  order  of  these  operations  being  the  reverse  of 
that  sometimes  employed  in  practice.  These  results  are  of  considerable 
importance  in  view  of  the  great  preponderance  of  cyclostationary  signals 
in  communication  systems  (as  emphasized  throughout  Chapter  II).  Specific 
examples  which  are  worked  out  in  detail  include,  among  others,  signals 
which  are  amplitude-modulated,  frequency-shift-keyed,  pulse-amplitude- 
modulated,  frequency-division-multiplexed,  and  time-division-multiplexed. 

Before  proceeding,  we  briefly  discuss  the  origin  of  the  above 
proposition  on  improved  performance:  In  Section  S  of  Chapter  II,  it  was 
shown  that  any  cyclostationary  process  can  be  reduced  to  a  stationary 
process  by  randomizing  the  phase  (or  time-origin)  of  the  process  (Theorem 
(2-15)),  and  that  the  resultant  autocorrelation  function  is  precisely  the 
same  as  the  function  obtained  by  averaging,  over  one  period,  the  original 
autocorrelation  function  of  the  cyclostationary  process  (Theorem  (2-16)). 

This  has  long  been  known  (as  brought  out  in  the  historical  notes  of 
Section  3  of  Chapter  I)  and  indiscriminantly  used  to  simplify  analyses 
dealing  with  cyclostationary  processes.  In  fact  most  systems-analysts, 
in  solving  for  optimum  filters  for  cyclostationary  signals,  use  the 
stationarized  model  for  their  cyclostationary  signals,  and  come  up 
with  optimum  time-invariant  (non-coherent) 

25  To  my  knowledge,  the  only  other  result  of  this  nature  was  obtained  by 
Brelsford  (19,20].  He  experimentally  verified  the  fact  that  pure  predictions 
of  eye  lostationary  meteorological  data  based  on  autoregressive  techniques 
which  employ  periodically  time-varying  coefficients  can  be  superior  to 
similar  predictions  obtained  with  constant  coefficients. 
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filters.26 

There  are  two  dominant  Motives  for  this  approach: 

(1)  In  the  past,  time-varying  systems  were  considerably  more  difficult — 
if  not  totally  impractical — to  implement  compared  with  time-invariant 
systems,  so  that  even  if  an  optimum  periodically  time- varying  filter 

was  identified,  it  probably  couldn't  (or  wouldn't)  be  built. 

(2)  Solutions  for  optimum  (noncausal)  filters  for  stationary  processes 
are  very  simple  to  obtain,  in  direct  contrast  to  the  complete  lack  of 
any  general  solution  methods  for  optimum  (noncausal)  filters  for  non¬ 
stationary  processes.  (The  appropriateness  of  the  noncausal  optimum 
filter  for  communication  problems  will  be  discussed  in  Section  3.) 

Thus,  the  tack  taken  in  the  past  is  very  understandable.  However, 
with  the  advent  of  integrated  circuit  technology,  time-varying  systems— 
particularly  periodically  time-varying  systems--are  becoming  increasingly 
easier  to  implement.  In  fact,  some  such  implementations  are  even  more 
attractive  in  terms  of  design-and-construction  effort  and  cost  than 
classical  time-invariant  implementations  based  on  resistors,  capacitors, 
and  inductors.  With  this  new  technology,  the  question  of  improvements 
in  performance  available  through  recognition  of,  and  design  for,  the 
cyclic  character  of  cyclostationary  signals  becomes  very  interesting  and 
practical --and  challenging,  in  light  of  the  general  lack  of  appropriate 
solution  methods. 


26  The  most  predominant  exception  to  this  statement  is  the  coherent 
demodulator-filter,  for  AM  signals,  which  has  been  derived  in  various 
ways  by  many  investigators. 
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2.  The  Orthogonality  Condition 

The  necessary  and  sufficient  condition  which  implicitly  specifies 
the  optima  (linear)  filter  was  derived  for  the  case  of  stationary 
ergodic  processes  and  an  infinite-menory  filter  by  Wiener  [56]  in  the 
1940' s.  In  1950,  the  condition  was  derived  for  the  case  of  stationary 
ergcdic  processes  and  a  finite  memory  filter  by  Zadeh  and  Ragazzini  [57], 
and  in  1952  for  the  general  case  of  nonstationary  processes  by  Booton  [58]. 
These  classical  results  have  been  rederived,  modified,  and  generalized 
by  a  number  of  authors  using  various  tools  of  analysis  such  as  the  calculus 
of  variations,  directional  derivative  and  gradient  techniques  in 
functional  analysis,  the  Hilbert-space  projection  theorem,  and  others. 

We  will  present  here  a  short  derivation  of  a  very  general  form  of  the 
necessary  and  sufficient  condition  for  optimality  by  using  the  projection 
theorem  stated  below  [29,59-61]: 

Projection  Theorem:  Let  H  be  a  Hilbert  space,  and  S  a  closed  subspace 
of  H.  For  every  element  h  e  H  there  exist  a  unique  element  h  e  S  such 
that 


||h-h||H  <  | J h-s | |H  V-  s  e  S,  s  t  h  (4-1) 

and 

(h-h,s)H  =0  V-  s  e  S.  (4-2) 

In  this  theorem  h  is  called  the  orthogonal  projection  of  h  onto 
S,  and  Eq.  (4-2)  is  called  the  orthogonality  condition.  In  words,  this 
condition  states  that  the  difference  between  h  and  its  orthogonal 
projection  h  is  orthogonal  to  all  elements  in  the  subspace  S  onto  which 
h  was  projected. 
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A  simple  example  of  this  theorem  is  illustrated  in  Fig.  (4-2) 
where  we  have  chosen  the  two-dimensional  Euclidean  space-  as  our  Hilbert 
space.  In  this  space,  the  theorem  is  obvious  from  geometrical 
considerations . 

Now,  this  theorem  can  be  applied  to  many  optimization  problems 
simply  by  identifying  the  quantity  to  be  minimized  as  the  norm  of  the 
difference  between  a  given  element  h  and  a  variable  (adjustable) 
element  s  in  an  appropriate  Hilbert  space  H,  and  by  identifying  the 
constraints  on  the  variable  element  as  equivalent  to  the  restriction  of 
that  element  to  an  appropriate  subspace  S  of  the  Hilbert  space  H.  For 
the  least-mean-square  (LMS)  estimation  problem,  we  set  up  a  separate 
(independent)  optimization  problem  for  each  value  of  the  time-index: 

For  any  specific  value  of  time,  say  t,  we  identify  H  as  HRV  the  space 
of  finite  mean-square  random  variables  (defined  in  Section  1  of 
Chapter  II),  and  h  as  the  transmitted  signal  process  x  evaluated  at 
time  t,  and  s  as  the  output  of  the  estimator  which  is  to  be  optimized, 
so  that  minimizing  the  mean-squared  estimation-error  is  the  same  as 
minimizing  the  norm 

I |h-s 1 |2  =  | |x(t)-s(t)| |2  =  E{ |x(t)-s(t) | 2>,  teT 

RV  0 

subject  to 

s(t)  =  £(t)*[y] 

where  y  is  the  observed  (received)  signal  process,  and  where  is 


one  element  of  a  set  of  linear  functionals,  the  collection  of  which 
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comprises  the  estimator  transformation  L  -  ;  t  e  Tq).  Now,  in 

order  to  apply  the  projection  theorem,  we  need  only  identify  the 
subspace  c:  such  that  the  restriction  of  L  to  a  "permissable" 

set  L  is  equivalent  to  the  restriction  of  s(t)  to  the  subspace  for 
all  values  of  t  e  TQ.  We  first  give  a  precise  definition  of  the  term 
"permissable" : 

DEFINITION:  L  =  {2^}  is  defined  to  be  a  permissable  set  of  linear 
transformations  \.r.t.  the  observed  process  y,  and  P  =  {Ik  (•»•)} 
the  permissable  set  of  kernels  (impulse-response  functions),  if  and 
only  if: 

1.  The  sets  Pt  of  kernel -sections  {h^ (t , • ) } ,  corresponding  to  the 
linear  functionals  form  Banach  spaces  with  the  L1  (-»,“>)-  norm; 

1. e.,  the  sets  P^  form  linear  vector  spaces  which  are  complete  w.r.t. 
the  norm; 

CM 

I |h±Ct,-) | |Li  =  /  |h± (t,x) |dx  <  ». 

•  00 

2.  If  y  contains  a  non-zero  white  component,  then  the  kernel -sections 
form  Banach  spaces  with  the  L2 (-»,«) -norm: 

00 

I |hi(t,*) I |L2  =  /  h?(t,i)dT  <  «. 

•  00 

Note  that  P  is  also  a  linear  vector  space,  and  its  elements 
are  impulse-response  functions  of  "bounded- input-bounded-output" 
stable  systems  [52]  (because  of  the  finite  L1-  norm).  Note  also  that 
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although  the  finite  L2-norm  constraint  may  seem  unnecessarily 
restrictive — in  the  sense  that  it  excludes  systems  with  direct 
connections  from  input  to  ou*:put — it  is  a  practical  constraint  since 
it  is  necessary  in  order  that  the  estimate  of  a  finite  mean-square  process, 
based  on  observations  containing  a  white  component,  be  a  finite  mean- 
square  process. 

Now,  we  have  the  following  theorem: 

THE0REM(4-1) :  Let  L  be  the  permissable  set  of  linear  transformations 
corresponding  to  an  arbitrary  permissable  set  of  kernels  P,  and  let  y 
be  a  random  process  which  can  be  finite-mean-square,  or  white,  or  the 
sum  of  both  (assumed  uncorrelated).  Then,  the  restriction  of  the 
transformation  L  to  the  set  L  is  equivalent  to  the  restriction  of  the 
random  variables  {s(t);  t  e  T  }  to  respective  closed  subspaces  St  of 
the  Hilbert  space  H^,  where 

s(t)  =  £ct)-[y],  U(t);  t  e  To>  =  L  t  L, 

Proof:  By  definition  of  L,  P,  P  ,  we  have  the  fac"  that  for  any 
transformation  L  e  L,  there  exist  kernel-sections  h^(t,*)  e  P^  such  that 

00 

s(t)  =  I^«[y]  =  /  hi(t,T)y(x)di 


V-  t  e  T  . 
o 
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Now, 

00 

E{s2(t)}  =  //  h£ (t , t)hi (t , o) k  (t , o)drdo  ' 

.00  / * 

00  00 

=  //  l»i(t,T)hi(t,o)i; y  (T,a)dxdo  +  A  /  h?(t,T)dt 

-oo  yy  oo  1 

where  kyy  is  the  autocorrelation  function  of  the  finite  mean-square 
component  of  y,  and  A  is  the  PSD  of  the  white  component.  Thus,  we  have 

oo 

E{s2(t)}  <  max  k  (t,t)[  /  |h.(t,o) |da]2+A  /  h?(t,a)do  <  ~ 

x  y*  -00  _00  1 


v-  t  e  T 

o 

since  h^t,*)  e  L1  (-»,»)  and  since  h^t,*)  e  L2(-»,»)  if  A  ^  0. 

Hence,  s(t)  e  H^y  ¥■  t  e  Tq.  Also,  since  P^  is  a  banach  space  with  both 
L1  and  L2  norms,  its  easily  shown  that  the  set  of  all  s(t)  generated  from 
y  using  all  the  elements  of  P  is  a  closed  linear  vector  space.  This 
space  of  finite  mean-square  random  variables  is,  then,  a  closed 
subspace  of  HRy 

QED 

Using  the  result  of  this  last  theorem,  and  the  projection 
theorem,  we  obtain  the  following  theorem  which  provides  a  necessary 
and  sufficient  condition  for  optimum  linear  filtering: 
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THEOREM (4-2) :  Let  P  be  any  permissable  set  of  kernels.  The  filter 
with  impulserresponse  function  h^  ( • ,  • )  minimizes  the  ae^n-squared  error 

e(t)  =  E{(x(t)-x(t))2}  V-  t  e  Tq 

where 

00 

x(t)  =  /  h  (t,x)y(x)dx,  (4-3) 

_00 

subject  to  ho(*,*)  e  P,  if  and  only  if 

00  CO 

//  h .  (t,x)h  (t,a)k  (x,a)dxda  =  /  h  (t,x)k  (x,t)dx  (4-4) 

-oo  1  °  yy  _oo  1  y* 

for  every  h^ (• , • )  e  P,  and  for  every  t  e  T  ,  and  the  minimum  mean-squared 
error  is 

00 

eo(t)  =  kxx(t,t)  '  /  h0(t'X)kyx(x»t)dx.  (4-5) 

Proof:  From  the  projection  theorem  (Eq.  (4-2)),  x(t)  is  the  LMS 
estimate  of  x(t)  e  HRy,  V-  t  e  Tq,  if  and  only  if 

E{(x(t)-x(t))s(t)}  =  0  V-  s(t)  e  S tc  HRV>  V-  t  e  T.. 

But,  from  Theorem  (4-1), 

00 

st  =  {  /  hi(t,x)y(x)dx;  h^t,*)  e  Pt)  V-  t  e  Tq.  (4-6) 

»00 

Using  this  representation  for  St>  and  the  formula  (Eq.  (4-3))  for  x(t) 
in  the  above  orthogonality  condition  yields  the  necessary  and  sufficient 
condition  of  Eq.  (4-4).  Furthermore,  the  minimum  mean-squared  error  can 
be  written  as 

eo(t)  =  E{ (x(t)-x(t))x(t) }  -  E{ (x(t)-x(t))x(t) } , 
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and  since  x(t)  e  S^,  then  the  last  term  is  zero.  Now,  using  Eq.  (4-3) 
we  obtain  the  formula  of  Eq.  (4-5). 

QEO 

This  necessary  and  sufficient  condition  of  Eq.  (4-4)  is  a 
considerable  generalization  of  the  integral  equation  frequently,  but 
often  inappropriately,  referred  to  as  the  Wiener-Hopf  equation  for 
optimum  filters.  Several  specific  examples  are  given  below: 

(1)  "Infinite-dimensional"  filters:  If  the  linear  vector  space  P  of 
permissable  kernels  is  infinite-dimensional,  then  we  will  call  the  optimum 
filter  in  P  an  infinite-dimensional  filter.27  Typical  constraints  which 
lead  to  infinite-dimensional  P's  are  constraints  on  memory  length.  For 
example,  we  may  define  P  such  that 


h.  (t,t)  =  0,  V-  t, t  3  t-T  < 


T 


or  t-T  >  T 


for  every  h.  (♦,♦)  e  P,  where  is  the  memory  length  of  the  filter 
(amount  of  past  input  used  to  obtain  the  current  output),  is  the 
reverse  memory  length  of  the  filter  (amount  of  future  input  used  to 
obtain  the  current  output — also,  the  amount  of  delay  necessary  to  make 
the  filter  causal),  and  T^,  are  the  initial  and  final  times  which  define 
the  interval  over  which  the  filter  is  to  operate.  In  this  case,  our 
general  condition  for  optimality  becomes: 


bit) 

/  h  (t.s)k  ;.-,o)do  =  k  (x,t), 
a(t)  y 


V-  T  e  [a(t)  ,b(t)] 
¥•  t  e  [T.,Tf] 


(4-7) 


27  Note  that  finite  order  linear  dynamical  systems  are  sometimes  referred 
to  as  finite-dimensional  systems  [52],  but  are,  under  the  above  definition, 
infinite-dimensional.  Furthermore  note  that  the  finite  dimensional  filters 
defined  here  do  not  necessarily  have  finite  dimensional  range  spaces. 
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where 

a(t)  -  maxiT^t-T^} 

b(tj  -  fliin{Tf,t+Trm}.  (4-7) 

As  a  specific  example,  if  we  choose  -T\  =  **£  =  =  “»  Trm  =  0,  then 

P  is  the  set  of  all  causal  filters,  and  if  y  and  x  are  jointly  WSS,  then 
the  above  equation  becomes  a  Wiener-Hopf  equation. 

(?)  "Finite-dimensional"  filters:  If  P  is  spanned  by  a  finite  set  of 
kernels,  then  we  call  the  optimum  filter  in  P  a  finite-dimensional  filter. 
For  example,  if  P  is  M-dimensfonal ,  then  every  impulse-response  function 
in  P  can  be  expressed  as  a  linear  time-varying  combination  of  M  basis 
kernels: 

M 

h(t,x)  =  l  a. (t)h. (t.x), 
i=l  1 

where  the  a-priori  choice  of  the  basis  kernels  defines  the  vector  space 
P.  Notice  that  the  scalars  in  P  are  not  real  numbers  as  in  P^,  but 
rather  real  time-functions.  This  is  a  result  of  the  way  in  which  P  was 
constructed  from  (P^;  t  e  Tq). 

For  finite-dimensional  filters,  our  optimality  condition  becomes: 

M  “ 

I  <**ftlt)  //  h.  (t,x)h  (t,a)k  (T,o)dTdo 
i=l  -«©  J  yy 

00 

=  /  h  (t,o)k  (a,t)da  (4-8) 

-oo  J  y 

for  every  t  e  T  ,  and  for  j  =  1,2,...,M,  where  (a^Q(t)}  is  the  optimum 
set  or  weighting  functions.  Hence,  the  optimum  M-dimensional  filter 
has  impulse-response  function 
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i 

M 

h  (t,x)  =  l  aA  (t)h.(t,T), 

i=l 

t 

where  the  M-vector  of  weights  a  (t)  is,  from  Eq.  (4-8)  f 


0,(1)  =  [A(t)j  "^b(t) 

where  A(t)  is  the  M  x  M  matrix  with  elements: 

CO 

a.. ft)  =  ff  h. (t,x)h. ft.d)k  fx.aldxdo 

ij  ”  i  r  yy 

and  b(t)  is  the  M-vector  with  elements:. 


b.(t) 


/  h  (t,x)k  (x,t)dx. 
oo  y 


(4-9) 


(4-10) 


(4-11) 


Note  that  the  basis  filters  can  be  chosen  to  have  arbitrary  memory 
length,  and  reverse  memory  length;  they  can  be  chosen  to  be  arbitrarily 
time-varying  filters,  or  causal  time- invariant  filters.  They  need  only 
be  ac'ible  filters  as  defined  in  the  definition  of  P.  Specific  choices 
can  be  bas^d  on  physical  considerations  of  implementation,  and  on 
considerations  o,f  efficiency  (minimization  of  the  dimension  necessary 

I 

to  reduce  the  mean-squared  estimation-error  below  a  prescribed  level.) 

A  formal  realization  of  an  arbitrary  M-dimensional  filter  is  shown  in 
Fig.  M-3)  as  a  parallel  connection  of  M  basis  filters  each  followed  by 

'  i 

a  multiplier. 

In  this  chapter  we  will  be  primarily  concerned  with  infinite 
dimensional  noncausal  filters  (with  infinite  forward  and  reverse  memory 
length)  on  the  doubly  infinite  interval  i.e.,  we  will  make  the 

choice  of  parameters:  Tm  =  r  -T  =  T.  =  ~,  so  that  the  general 
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necessary  and  sufficient  condition  for  optimality  becomes: 

CO 

/  h  (t,s)k  (s,r)ds  =  k  (r,t),  V- t,t  e  (-»,«),  (4-12) 

_cb  u  yy 

and  the  minimum  mean-squared  estimation  error  is  given  by 

CO 

e  (t)  =  k  (t,t)  -  /  h  (t,s)k  (s,t)ds,  V- t  e  (-«,»).  (4-14) 

O  aA  -co  u 

Before  proceeding,  we  comment  on  our  choice  of  the  "integral - 
equation  approach"  to  our  linear  least -mean- square  estimation  problems, 
as  opposed  to  the  "differential-equation  approach",  which  incidentally 
can  also  be  developed  from  the  Projection  Theorem.  The  type  of  LMS 
estimation  problems  considered  here,  like  many  other  communication-systems - 
analysis  problems,  are  most  easily  formulated  in  terms  of  integral 
transformations  and  impulse-response  functions,  whereas  typical  control- 
systems-analysis  problems  (and  some  communications  problems)  are  most 
easily  formulated  in  terms  of  differential  transformations  and  state- 
evolution  equations.  For  example,  the  various  communication-signal 
formats  and  models  for  cyclostationary  communications  signals  which  were 
presented  in  Chapter  II  were  developed  in  terms  of  subjecting  elementary 
information  bearing  random  processes  (including  multidimensional 
continuously  indexed  processes  as  well  as  quantized  discretely  indexed 
sequences)  to  systems  (including  transducers,  modulators,  multiplexors, 
and  various  signal  processors)  which  were  conveniently  characterized  as 
integral  transformations  with  easily  specified  impulse-response  functions. 
Equivalent  state-equation  descriptions  of  these  signals  and  systems 
would,  in  most  cases,  be  very  cumbersome  or  impossible  due  to  noncausality 
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and/or  infinite-dimensionality.  For  example,  the  concept  of  an  ideal 
bandlimiting  filter  is  indispensible  in  many  analyses  of  communications 
problems,  (including  data-sampling  and  frequency  division  multiplexing), 
yet  is  not  amenable  to  state-variable  description  since  the  filter  is 
noncausai;  and  the  concept  of  an  ideal  delay-line  is  of  great  utility  in 
many  communications  problems  (including  filtering  of  pulsed  signals,  and 
time-division  multiplexing),  but  is  an  infinite  dimensional  (order) 
differential  system  and  is  therefore  not  amenable  to  state-variable 
description;  and  time- (and  other  index)  scale  transformations  (which  are 
useful  for  characterizing  scanning  operations,  Doppler  effects,  signal 
compression,  etc.)  are  easily  described  with  impulse-response  functions, 
but  are  formidable  with  state  equations  due  to  infinite-dimensionality; 
and  so  on. 

Now,  since  the  state-variable  approach  to  LMS  estimation  problems-- 
Kalman  filtering--requires  that  all  signals  or  processes  be  generated 
from  white  noise  via  a  causal  finite-order  differential  system  [60,61], 
then  it  is  of  severely  limited  value  for  tne  problems  of  concern  here. 
Furthermore,  since  we  are  specifically  interested  in  noncausai  filtering 
(with  infinite  reverse  memory!,  then  the  Kalman  filter  (which  must  be 
causal  with  either  zero  or  finite  reverse  memory)  is  of  little  value  to 
us.  In  addition,  "finite-dimensional"  optimum  filters  (not  necessarily 
finite  order  differential  systems)  are  easily  solved  for  with  the 
integral -equation  approach  as  discussed  in  this  section,  yet  are  excluded 
from  the  differential  equation  approach.  Finally,  we  mention  that  our 
approach  leads  to  explicit  solutions  for  optimum  filters  and  performances, 
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and  useful  insight  into  optimum  time- varying  signal  processors,  whereas 
the  Kalman  filtering  approach  leads  to  the  Riccati  matrix  differential 
equation  which  almost  always  must  be  solved  numerically. 

One  last  comment:  All  results  derived  in  this  chapter  using 
minimum-mean-squared-error  as  an  optimality  criterion  are  also  valid,  when 
the  optimality  criterion  is  minimum-error-variance  provided  that  all 
processes  involved  are  replaced  with  their  centered  versions;  e.g.,  x 
replaced  with  x  -  m^. 
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3.  Optimum  Time- Invariant  Filters  for 
Stationary  Processes 

The  optimum  noncausal  filter  for  wide-sense  stationary  processes 
is  very  simple  to  solve  for,  and  yet  is  of  significant  theoretical 
and  practical  value.  Three  of  the  most  outstanding  reasons  for  its 
utility  are: 

(1)  The  explicit  solution  gives  useful  insight  into  the  nature  of 
optimum  filters. 

(2)  A  delayed  version  of  the  optimum  noncausal  filter  can  often  be 
closely  approximated  by  a  causal  filter, 

(3)  The  mean-squared  estimation-error  which  results  from  the  optimum 
noncausal  filter  is  a  useful  performance  bound  since  no  causal  filter 
can  possibly  perform  any  better. 

The  second  fact  above  has  useful  application  in  communication 
problems  since  communication  systems  are  often  tolerant  of,  or 
insensitive  to,  the  incorporation  of  a  delay  in  the  receiver.  (Note, 
however,  that  this  is  far  less  frequently  true  for  control  problems, 
since  control  systems  are  often  feedback  systems,  and  are,  by  nature, 
very  sensitive  to  the  incorporation  of  delays,  and  can  in  fact  become 
unstable  when  relatively  small  delays  are  included  in  the  feedback  loop.) 

With  this  as  motivation,  we  proceed  to  derive  the  optimum  noncausal 
filter.  Assuming  that  the  transmitted  and  received  signals  are  WSS,  the 
necessary  and  sufficient  condition  for  optimality  (Eq.  (4-12))  can  be 


written  as 
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/  kyy(T-s)h<j(t,s)ds  =  k^Cr-t)  V- t,i  e  (-»,»). 

But  the  solution  to  this  equation  depends  only  on  the  difference 

of  its  two  arguments.  Using  this  fact,  and  making  a  change  of  variables 

results  in  the  equivalent  condition  for  optimality: 

00 

/  k  (t-r)h  (x)dr  =  k  ft)  V-  t  e  (-“,“), 

_oo  yy  y 

which  is  a  convolution.  Hence,  the  Fourier  transform  of  ho~-the 
transfer  function  for  the  optimum  filter — is  given  explicitly  by  the 
formula 


H  (f)  =  K  (f)/K  (f).  (4-14) 

o  xy  yy 

Using  this  formula  in  Eq.  (4-13)  and  employing  Parseval's  relation  [2] 
yields  the  following  formula  for  the  minimum  mean-squared  estimation-error: 


«  K  (f)K  (f) 

e  (t)  -  /  -  -x-x  •  -  y.V— 


K  (f) 

yy 


|Kxy(f)l2 

— - df  V-  t  e  (-“,“) 


(4-15) 


where  K  is  the  cross  PSD  for  y  and  x,  and  K  ,  K  are  the  PSD's 
yx  yy  xx 

for  y,  x. 

If  we  consider  the  special  class  of  problems  where  the  channel  is 
composed  of  a  time- invariant  dispersion  with  transfer  function  G  and 
additive  noise  n,  assumed  uncorrelated  with  the  signal  x,  then  the  optimum 
filter  has  transfer  function  given  by 


G*(f)K  (f) 

H  (f)  =  - — - 

0  |G  (f)  1 2K  (f)  +  K  (f) 

1  v  J  1  xx  nn 

where  K  is  the  PSD  for  the  additive  noise. 


nn 

referred  to  as  the  Wiener  filter,  results  in 


(4-16) 

This  filter,  frequently 
a  minimum  mean-squared 
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estimation-error  which  is  given  by  the  formula 

CO  00 

eo  =  /  Kxx(f)|l-G(f)Ho(f)|2df  +  /  Knn(f)|HoCf)|2df.  (4-17) 

_0D  —CO 


In  this  expression,  the  first  term  represents  error  due  to  imperfect 
compensation  for  channel  dispersion,  while  the  second  term  represents 
error  due  to  the  filtered  noise  remaining  at  the  output.  A  more  compact 
formula  for  the  error  which  results  directly  from  Eq.  (4-15),  is 


e 

o 


K  (f)K  (f) 
nn  xx 


|G(f)|2K  (f)  +  K  (f) 

'  v  1  xx  nn 


df 


and  reduces  to 


(4-18) 


°°  N  K  (f) 

f  O  XX 

e  =  I  - 

o  J 


N  +  K  ff) 

O  XX 


df  =  N  h  (0) 
o  o 


(4-19) 


when  there  is  no  dispersion  (G(f)  =1)  and  the  additive  noise  is  white 
with  PSD  equal  to  Nq.  Although  Eq.  (4-19)  is  simple  in  form,  it  can 
present  difficulties  in  terms  of  numerical  computation  because  of  the 
infinite  interval  of  integration.  With  this  in  mind,  we  present  th? 
following  error  bounds  which  require  evaluation  of  an  integral  only  on 
a  finite  interval: 


THE0REM(4-3) :  The  minimum  mean-squared  estimacion-error  which  results 
from  the  Wiener  filter  for  the  dispersionless  channel  is  contained 
within  the  ±  1.5  dB  bounds: 

f 

o 

B/2  i  e  <  B,  B  =  k  (0)  -  2  /  [K  (f)  -  K  (f)Jdf  (4-20) 

—  u—  a  a  xx  nil 

provided  that  the  signal  PSD  dominates  the  noise  PSD  for  all  frequencies 
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less  than  fQ,  and  vice-versa  for  all  frequencies  greater  than  f  : 


K  (f) 
xx  1 


2  lfl  J  f„ 

5  *„»<«>  lfl  2  fo 


Proof:  By  hypothesis. 


and 


■f0  Kxx<f>  »  Kxx<« 


fQ  f 

■  J  - 2  / 

o  u 


f  K  (f)  +  K  (f) 
o  nnv  xx  ' 


"  -  K  (f)K  (f) 

/k  m«  =  2  /  — ss — *?_L  df 

fo  fo  Kn„(f)  * 

"  K  f£)K  (f) 

<2  /  - df  5  2  /  K  (f)df 


fo 


XX 


XX 


Adding  these  two  strings  of  inequalities,  and  using  the  fact  that 


2  J  Kxxtf)df  -  kxx(0)  -  2  J  Kxx(f)df, 


yields  the  bounds  of  Eq.  (4-20). 


df 


QED 


4.  Optimum  Time- Invariant  Filters  for 
Cyclostationary  Processes 

As  discussed  in  the  introductory  section  of  this  chapter,  an 
optimum  (in  a  sense  to  be  defined  below)  time-invariant  filter  for 
cyclostationary  processes  can  be  obtained  simply  by  replacing  the  given 
transmitted  and  received  T-CS  processes  x,  y  with  their  stationary 

f\j  O/ 

versions  x,  y  (Theorem  (2-15) )--which  have  realizations  of  the  form 

%  'b 

x(t)  =  x(t+9),  y(t)  =  y(t+0),  where  0  is  a  random  phase-variable  which 
is  uniformly  distributed  over  the  period  [-T/2,T/2] --and  then  solving 
for  the  optimum  filter  for  these  stationary  processes.  The  auto-  and 

%  *b 

cross-correlation  functions  for  these  stationary  processes  x,  y  are 
(Theorem  (2-16)): 

i  ,T/2 

kyy(t)  =  t  JT/2  kyy(t*T’t)dt 

T/2 

kfx)  =  f  /  k  (t*T,t)dt  (4-21) 

xy  1  -T/2  ™ 

so  that  the  optimum  time-invariant  filter  has  transfer  function  given 
by  (Eq.  (4-14)) 

»0m  ■  Vf)/Vf)>  (4‘22) 

where  ft  ,  ft  are  the  Fourier  transforms  of  ft  ,  ft  and  are  therefore 
xy  yy  xy  yy 

'Xj  % 

PSD's  for  the  stationary  processes  y.  However,  the  mean-squared 
estimation-error  resulting  from  this  filter  will  not  be  given  by  Eq. 

'b 

since  is,  by  definition. 


(4-15) 
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e  ft)  =  Effxft)  -  x(t))2}  t  Effxft)  -  x(t))2} 


and  is,  in  fact,  a  T-periodic  function  rather  than  a  constant.  On 

*\j 

the  other  hand,  the  time-averaged  value  of  eQ(t)  is  indeed  given  by 
the  formula  in  Eq.  (4-15): 


<v  i  f  /  *ct)dt  =  /  — — - & - 

T  -T/2  0  -»  5f  (f) 


df.  (4-23) 


Furthermore,  the  suboptimura  filter  with  transfer  function  (sub- 
because  it  ignores  the  cyclic  character  of  the  cyclostationary  processes) 
is  actually  optimum  subject  to  the  constraint  of  time-invariance;  i.e., 
it  minimizes  the  time-averaged  value  of  the  periodically  time-varying 
mean-squared  estimation-error.  This  result  is  stated  here  as  a  theorem: 


THEOREM (4-4) :  The  optimum  filter,  given  by  Eq.  (4-22),  for  the 

O i  ^ 

stationarized  versions  x,  y  of  the  cyclostationary  processes  x,  y  is 
time -invariant  and  is  the  same  as  the  filter  which  minimizes  the  time- 
averaged  value  of  che  periodic  mean-squared  estimation-error  <e>,  and  the 

f\j 

minimum  error  <eQ>  is  given  by  Eq.  (4-23). 

Proof:  By  definition, 


<e>  =  ^  /  E{[x(t)  -  /  h(x)y(t-x)dx]2>dt 

-T/2 

T/2  a> 

=  4  /  [k  (t,t)-2  /  h(x)k  (t,t-x)dx 


+  //  h(o)h(x)k  (t-o,t-x)dodx]d" 

_  CO  *  ♦ 
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oo  go 

=  &xx(0)  -  2  /  h(T)fc  (t-(t-T))dT  +  JJh(0)h(T)&  ( (t-a) - (t-T) )dadr 

“  *  .OO  '  .00  *  * 

00 

=  E{[x(t)  -  /  h(T)y(t-i)dT]2}. 

_oo 

But  this  last  expression  is  the  mean-squared  estimation-error  for  the 

r\j  e\, 

problem  with  transmitted  and  received  signals  x,  y  and  filter- impulse- 

A/ 

response  function  h,  and  is  therefore  minimized  when  h  =  hQ,  the  inverse 
Fourier  transform  of  given  by  Eq.  (4-22).  Furthermore,  the  minimum 
value  of  this  time-averaged  error  is 

<eQ>  =  E{  (x(t)-x(t))2}  =  E{[x(t)-  /  lio(T)y(t-T)dtJ2} 

.OO 


and  is  therefore  given  by  Eq.  (4-23). 


QED 


If  we  consider  the  special  class  of  problems  where  the  channel 
consists  of  a  time- invariant  dispersion  with  transfer  function  G,  and 
additive  WSS  noise  with  PSD  Kfin,  then  paralleling  the  results  in  the 
preceding  section,  we  have  the  following  formula's  for  the  transfer 
function  for  the  optimum  time-invariant  filter,  and  for  the  resulting 
minimum  time-averaged  error. 


ft  m  - 


G*(f)Kxx(0 


% 

<e  >  = 
o 


|G(f)P\x(f)  *  Kln(f) 
K  (f)K  (f) 


I 


|GCf)|2lfxx<f)  * 


nn 


(^-24) 

(4-25) 


and  for  white  noise  and  no  dispersion 


% 

K  (1) 

U  (fj  =  - - -X-~ - 

°  K  (f)  +  N 

XX  o 


(4-26) 
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„  -  N  (f) 

<e  >  =  /  - df  =  N  h  (0).  (4-27) 

°  J  *  (f)+N  00 

These  last  four  formulas  fully  characterize  the  optimum  time- 
invariant  filters  and  corresponding  time-averaged  estimation-errors 
for  the  specific  illustrative  problems  that  we  will  consider  in  the 
last  section  (6)  of  this  chapter. 


5.  Optimum  Time-Varying  Filters  for 
Cyclostationary  Processes 

a)  Introduction .  In  this  section  we  present  a  number  of  general 

methods  of  solution  for  optimum  time-varying  noncausal  linear  filters 

« 

for  cyclostationary  processes.  In  contrast  to  the  optimum  time-invariant 
filters  presented  in  the  previous  section,  these  filters  are  not  constrained 
to  be  time- invariant  and,  in  fact,  minimize  the  mean-squared  estimation- 
error  for  every  value  of  time  thereby  taking  full  account  of  the  cyclic 
character  of  the  cyclostationary  processes.  As  claimed  in  the  introductory 
section,  these  optimum  per.!  dically  time-varying  linear  filters  result  in 
’ower  mean-squared  estimation-errors,  as  will  be  demonstrated  via  a 
number  of  specific  solutions  in  the  following  section. 

In  order  to  characterize  the  relative  performance  of  optimum 
time-invariant  and  optimum  time-varying  filters  with  a  single  number, 
we  choose  as  a  performance  index  I  the  ratio  of  the  minimum  time-averaged 
mean-squared  estimation-error  <eQ->,  resulting  from  the  optimum  time- 
invariant  filter,  to  the  time-averaged  value  of  the  minimum  mean-squared 
estimation-error  <eQ>  resulting  from  the  optimum  time-varying  filter: 
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(4-28) 


where  <eo>is  given  by  Eqs.  (4-23,25,27),  and  where  <eQ>  is  from  Eq.  (4-13), 


<e  >=  ^  /  (k  (t,t)  -  /  h  (t,s)k  (s,t)ds1dt 
u  ,-T/2  AA  u  /A 


(4-29) 


where  hQ  is  the  impulse-response  function  of  the  optimum  time-varying 
filter  and  is  the  solution  of  the  integral  equation  (Eq.  (4-12)): 

CD 

/  h  (t,s)k  (s,T)ds  =  k  (t,t)  ¥•  t,t  e  (-«,“).  (4-30) 

°  yy  y*- 

If  we  consider  the  special  class  of  problems  where  the  channel  is 
a  time- invariant  dispersion  with  impulse-response  function  g  and  additive  inde¬ 
pendent  WSS  noise  with  autocorrelation  function  k  ,  then  the  above  condition 
r  nn 

for  optimality  becomes 


/  ho(r.,s)[kjz(s,T)  +  knn(s,i)]ds  =  kzx(T,t)  V- t,t  e  (-“,“) 


chere. 


k  (t,T)  =  //  g(t-cr)g(j-s)k  (o,s)dods 

24  <»a 


k  (t,T)  =  /  g(t,o)k  (o,T)Jo. 

1.A  AA 


(4-31) 


For  the  case  of  white  noise  and  no  dispersion,  the  formula  for  the 
time-averaged  minimum  error  reduces  to 
N  1/2 

<e  >  =  =£  /  h  (t,t)dt 
o  T  Jx/2  o 


(4-32) 


where  the  optimum  impulse-response  function  is  now  the  solution  of  the 
integral  equation 
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/  h  (t,s)kvx(s,x)ds.  +  NQhQ (t,x) 

oo 


=  kxx(t»T)  *  t*T  E  (-“»“) 


(4-33) 


For  this  class  of  additive-white-noise  problems,  our  performance  index 
takes  the  form 


K (0) 


l  /  h  (t,t)dt 
-T/2  0 


(4-34) 


In  the  remaining  subsections,  we  present  four  different  methods  for 
solution  of  the  integral  equation  Eq.  (4-30)  (and  its  special  forms 
Eqs.  (4-31,33))  which  implicitly  specifies  the  optimum  filter  and 
minimum  estimation-error.  Although  these  solutions  are  not  as  simple  to 
obtain  as  were  the  solutions  for  optimum  time- invariant  filters,  they 
are  still  of  significant  practical  and  theoretical  value  for  the  same 
reasons  cited  in  Section  3: 

(1)  The  explicit  solution  gives  useful  insight  into  the  nature  of 
optimum  time-varying  filters. 

(2)  A  delayed  version  of  the  optimum  noncausal  filter  can  often  be 
closely  approximated  by  a  causal  filter. 

(3)  The  mean-squared  estimation-error  which  results  from  the  optimum 
noncausal  filter  is  a  useiul  performance  bound,  since  no  causal  filter 
can  possibly  perform  better. 


b)  Solutions  based  on  translation  series  representations.  We  begin 
by  considering  the  general  LMS  estimation  problem  where  the  channel  is 
arbitrary  but  where  the  transmitted  and  received  signals  are  jointly 
T-CS.  If  we  assume,  for  generality,  that  y  is  composed  of  a  colored 


i 
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component  w  and  an  uncorrelated  white  component  with  PSD  A,  then  the 

i  i 

optimality  Condition  of  Eq .  (4-30)  becomes  ■  > 

*  I  9 

00  f 

J  ^(t.sJk^CSj-o'ds  +  Aho(t,t)  =  kxw(t,x)  ¥■  t,T  e  (-»,»), (4-35) 

•  00  J 

* 

But  this  integral  equation  is  of  the  same  forn\  as  Eq.  (3-39)  of  Section 
2f  in  Chapter  III  (except  for  the  juxtaposition  of  the  kernels  hQ,  k^).  . 

•  i  .  ' 

Hence,  Theorem  (3-2)  gives  ys  the  following  solution: 


THEOREM (4-5) :  The,  impulse-response  function  for  the  optimum  filter  for  >  • 

i  •  i 

the  general  LMS  estimation  problem  where  the  transmitted  and  received  ! 

’  '  ■  '  „  i 

signals  x,  y  are,  jointly'  T-CS  and  where  y  is  composed  of  a  colored  component 
w  and  an  uncorrelated  white  component  with  PiSD  A  has  the  representation: 


‘  h(t,T).=  J"  7  Hpq  <j>  (t-iiT)<f*(s-,mT) 

0  „  m  ‘  n-m  p  '  qv  ' 

n,m=-«  p ,q=l  r  n 


(4-36,) 


where  { 4>  }  is  any  CON  set  on  L2[0,T]  and'where  the  sequence  of  matrices 

Pi  : 

{Hr}  is  given' by  the  inverse  z-transform  i  1 


:•  H  =  T  j  /  H(f)e^27irTf 

^  1  /  'VT' 


df 


-1/2T  , 

where  the  matrix  of  functions  H(f)  is  given  by 

I 

i 

H(f)  =  G(f)[D(f)  +  AT]"1  , 

i 

where  the, matrices  D;  G  are  the  z-transforms 

DCf.)  =  l  b  e-j2*rTf 


r=-c 


<±(f)  = .  I '  V 

r 


,  -j2irrTf 


'  (4-37) 


I  ! 


(4-38) 


(4-39) 


I 
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and  where  the  elements  of  the  sequences  of  matrices  D^,  are  given  by 
T 

^  =  //  kww Ct:) 4>q (x)dtdx 
T 

Glq  =  //  k  (t+rT,x)$*(t)<fr  (x)dtdT,  (4-40) 

r  ©  A"  P  4 

and  are  the  correlation  matrices  corresponding  to  the  translation  series 
representations  (TSR's)  for  x  and  w: 


where  A(f)  is  the  correlation  matrix  corresponding  to  the  TSR  for  x 
(defined  as  in  Eq.  (4-39,40)  for  w) . 

Proof:  The  validity  of  the  solution  Eq.  (4-36)  to  the  integral  equation 

Eq.  (4-35)  follows  directly  from  Theroem  (3-2)  of  Chapter  III.  The  formula 

Eq.  (4-43)  can  be  derived  from  Eq.  (4-29)  as  follows:  Substituting 

Eq.  (4-36)  for  h  ,  and  the  TSR's  for  k  and  k  into  Eq.  (4-29)  yields: 
o  xx  xy 
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<«  >  4  /  [  I  l  AlqjJt~n'T')<(t-mT') 

o  1  ~r  n-m  p  q 

0  n,m  p,q  r  ^ 


-mi  0P(t-nI)*:(s-,T)' 


n,m  p,q 


P 

»n  » 


I  l  Gn'  m'  *:.(t‘n’T)*  , (s-m'T)ds]dt 
n '  m '  p '  ,q •  11  1,1  P  4 


1  J  [ApP  -  J  J  rf*  G*®”*] 
T  1  o  L  L  m  mJ 
P 


m  q 


1/2T 


=  /  trace [A (f)  -  H(f)G* ' (f)]df . 

-1/2T 

Now,  substituting  Eq.  (4-38)  for  H(f)  yields  Eq.  (4-43) 


QED 


As  discussed  in  Section  2g  of  Chapter  III,  a  linear  system  whose 

impulse-response  function  has  an  order  TSR,  as  does  hQ  in  this 

theorem  (with  M  =  ®),  can  be  realized  with  the  structure  shown  in 

Figure  (3-9)  and  reproduced  here  as  Figure  (4-4a).  Although  the  number 

(M)  of  paths  in  this  optimum  filter  for  CS  processes  will  in  general  be 

infinite,  there  are  a  number  of  interesting  cases  where  M  is  finite,  as 

discussed  in  Sec.  2f  of  Chapter  III  and  as  illustrated  in  the  last 

section  of  this  chapter.  Furthermore,  since  any  CS  process  in  H..  (-“,“) 

Lb  1 ) 

can  be  artitbarily  closely  approximated  in  norm  by  a  finite  order  TSR 
(as  discussed  in  Section  2a  of  Chapter  III),  then  the  performance  of  a 
truncated  (to  M  paths)  optimum  filter  can  be  made  arbitrarily  close  to 
the  optimum  performance  by  choosing  the  finite  number  (M)  of  paths 
large  enough. 
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An  interesting  interpretation  of  this  optimum  filter  follows 

directly  from  the  recognition  that  the  bank  of  input  filters  and  switches 

comprises  the  process -resolution  operation  (as  discussed  in  Sec.  2c  of 

Chapter  III),  and  that  the  bank  of  output  filters  and  the  summing  device 

comprise  the  process-reconstruction  operation,  and  finally  that  the  transfer 

function  H(f)  (Eq.  (4-38))  for  the  matrix  of  sampled-data  filters  which  is 

inserted  between  the  resolution  and  reconstruction  systems  is  a  matrix 

version  of  the  transfer  function  HQ(f)  (Eq.  (4-14))  for  the  optimum  time- 

invariant  filter  for  stationary  processes,  in  that  G(f)  in  Eq.  (4-38)  is 

the  cross-correlation  matrix  for  the  representors  of  x,  y--paralleling 

K  (f)  of  Eq.  (4-14)--and  fD(f)  +  AI]  in  Eq.  (4-38)  is  the  autocorrelation 

xy 

matrix  for  the  representors  ot  y--paralleling  K^(f)  =  Kww(f)  +  A  of 
Eq.  (4-14).  Thus,  we  see  that  the  optimum  time-varying  filter  of 
Theorem  (4-5)  for  CS  processes  resolves  the  received  CS  process  y  into  a 
multiplicity  of  jointly  WSS  sequences,  subjects  these  to  an  optimum  time- 
invariant  sampled-data  filter,  then  reconstructs  from  these  filtered 
sequences,  the  optimally  filtered  continuous-time  estimate  x  of  the 
transmitted  signal  x.  Hence,  by  employing  TSR's  we  convert  the  problem 
of  IMS  estimation  of  a  scalar  CS  process  to  the  problem  of  LMS  estimation 
of  a  vector  of  jointly  WSS  sequences.  This  could  have  been  done  in  a 
non-rigorous  suboptimum  way  simply  by  periodically  sampling  the  received 
signal,  and  solving  for  the  LMS  estimates  of  the  corresponding  samples 
of  the  transmitted  signal. 

The  optimum  filter  derived  in  Theorem  (4-5)  can  also  be  realized 
with  the  alternate  structure  shown  in  Fig.  (4 - 4b)  where  the  filter- 
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sampler  combinations  at  the  input  have  been  replaced  with  multiplier- 
integrators,  and  where  the  impulse-driven  pulse-generating  filters  at 
the  output  have  been  replaced  with  pulse-amplitude  modulators,  and 
where  the  feedback-delay  device  (shown  in  Fig.  (3-3)),  which  has  an 
effect  equivalent  to  periodically  dumping  the  integrator,  has  been 
included  in  the  sampled  data  filter  by  modifying  its  transfer  function 
to  include  the  factor  1/(1  +  e  J^vfT^ 

As  discussed  in  Chapter  III,  some  CS  signals  which  are  used  in 
communication  systems  admit  finite  order  TSR's,  but  with  basis  functions 
which  are  not  necessarily  duration  limited  to  [0,T],  nor  even  orthogonal. 
With  this  in  mind,  we  present  the  following  extension  of  Theorem  (4-S): 


THE0REM(4-6) :  If  the  transmitted  signal  x,  and  the  colored  component  w 

til 

of  the  received  signal  y  admit  M  1  order  TSR's 


M  » 

x(t)  *  l  l  anpyt-nT) 
p=l  n=-“  y  * 

M 

“CO  -  I  *  8„Ct-nT) 


P=1 


np  p 


V-  t  e  (-00,00) 


(4-44) 


where  the  functions  in  the  sets  Up}.  { )  are  arbitrary,  then  the 
optimum  impulse-response  function  admits  the  representation 

M  « 

h  (t,x)  =  T  I  4>  (t-nT)O* (T-mT) 


P*q=l  n,m=- 


n-m  p 


(4-45) 


where  {H^  )  are  defined  as  in  Eq.  (4-37),  where 
n-m  n 


H(f)  =  G(f)[E(f)D(f)  +  A  1  ] " 1 


(4-46) 
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where  G,  D  are  defined  as  in  Eqs.  (4-39),  (4-41),  and  where  E  is  the 


2-transform  of  the  matrix  of  sequences 


E?qm  =  /  0*(t-nT)0n(t-nT)dt.  (4-47) 

n'm  -00  P  q 


Furthermore,  tne  time-averaged  value  of  the  minimum  mean-squared 
estimation-error  is  given  by  the  formula: 

1/2T 

<e  >  =  /  trace[B(f)A(f)-B(f)G(f)[E(f)D(f)+XI]"1E(f)G*'(f)]df  (4-4») 

0  -1/2T  ~  ' 


where  iB  is  the  z-transform  of  the  matrix  of  sequences 


=  /  <f>*  (t-nT) 4  (t-mT)dt .  (4-49) 

_  oo  *  * 


The  proof  of  this  theorem  follows  directly  that  of  Theorems  (3-2),  (4-5). 


The  optimum  filter  given  in  this  theorem  has  the  same  structure  as 


that  given  in  the  previous  theorem  and  shown  in  Fig.  (4-4),  except  that 

the  input  filters  have  transfer  functions  {0*}  rather  than  {$*}. 

P  P 

Another  interesting  solution  for  a  common  formulation  of  the  LMS 


estimation  problem  is  given  in  the  following  theorem: 


L. 


THEOREM(4-7) :  Consider  the  class  of  LMS  estimation  problems  where  the 

channel  consists  of  a  time-invariant  dispersion  with  transfer  function  G, 

and  additive  WSS  noise  with  PSD  K  ,  and  where  the  transmitted  signal  x 

nn 

is  a  T-CS  process  which  admits  an  Mtn  order  TSR: 


x(t) 


00  M 

■  l  l 

n=-°°  p=l 


a  <J>  (t-nT) 
np  p 
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For  this  class  of  problems,  the  impulse-response  function  for  the 
optimum  time-varying  filter  admits  the  M**1  order  TSR: 

00  M 

h  (t,x)  =  Y  I  Hpq  $  (t-nT)6*(x-mT)  (4-50) 

m,n=-°°  p,q=l  1 

where  0  is  the  inverse  Fourier  transform  of  the  function 

q 

e0(f)  =  G(f)*  (f)/Knn(f)  (4-51) 

and  where  the  sequence  of  M  x  M  matrices  (lO  is  given  by  the  inverse 
z-transform 

H  =  T  /  /  H(f)ej27rrTfdf 
r  -1/2T 

where  the  M  x  M  matrix  of  functions  H(f)  is  given  by 


H(f)  =  A(f)  [C(f )A(f)  +  Ij*5 


(4-52) 


where  the  matrices  A(f ) ,  C(f)  are  the  z-transforms 

CO 

,  V  a  -  j  2nrTi' 

A(f)  =  l  A  e  J 
-00 

C(f)  =  l  Cre‘j27TrTt 
r 

and  where  the  sequences  of  matrices  are  given  by  the  formulas 


APq  =  E{a 
r 


} 


Cpq  =  / 

r  ; 


(n+r)p  nq 
»  j G (f ) | 2$*(f)$  (f) 


K  (f) 
nn 


j2urTf ,  ~ 
cJ  df ; 


(4-53) 


Furthermore,  the  time-averaged  value  of  the  minimum  estimation-error 
is  given  by  the  formula 
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<e  >  =  /  trace[B(f)A(f)[C(f)A(f)  +  I_]_1]df 
°  -1/2T 


(4-54) 


where  .15  is  defined  in  Eq.  (4-49). 

Proof:  Substituting  Eq.  (4-50)  for  hQ  and  the  TSR  of  Eq.  (3-2)  for  k^x 
into  the  condition  for  optimality-Eq.  (4-31)  yields: 


y  i"  ♦  (t-nT)[  y  y  A^H1^',  /  e*f(a-n'T)$  ,(a-m'T)da* 

l  p  1  ,  ,L  ,  m'-m  n-n*  J  p'^  rq'  ' 

n,m  p,q=l  n’.m’  p’^q*  -°»  v  n 


$*(x-mT)  +  Hpq  [9*flk  ]  (t-mT)  -  Apq  J*(x-mT)]  =  0  V-  t,x  e  (-»,<») 
Yqv  1  n-m  q  nnJ  n-raTqv  1  '  v  ' 


where 


♦  (t)  -  l>  Sg3(t)  =  /  ♦fl(1‘-T)g(T)dx. 

*  *  „00  H 

But  since  9*Sknn  *  $*,  then  the  above  optimality  condition  can  be  written  as 


y  iii  i  Hpp,,cp;q,,Aq;q  +  Hpq 

L  .  I  i  *  n-n*  n’-m*  m’-m  n-i 

n,m  p,q=l  n' ,m'  p' ,q' 


-  APq  ]• 
m  n-m 


4>p (t-nT)$* (t-mT)  =  0  V-  t,x, 

and  will  be  satisfied  if  the  composite  coefficients  of  the  time-functions 
are  zero  for  every  value  of  n,m,p,q.  This  condition  on  the  coefficients 
can  be  written  (using  the  new  variables  i  =  m'  -  m,  j  -  n  -  n' ,  k  =  n  -  m)  as 


oo  M 


I  l  HPP'cp,q,.Aq,q  +  HPq-APq=0  Yk.q.p, 

i,j  =  -co  p',q<  =  l  J  1  k  k 


where 


Cpq  -  /  6* (o-rT)^  (o)do. 

r  p  q 


(4-55) 


But  this  quadruple  sum  is  simply  the  composition  of  a  discrete 
convolution  with  another  discrete  convolution  with  a  finite  matrix 
product  with  another  finite  matrix  product.  Recognizing  this,  and 
taking  z-transforms,  yields  the  equivalent  condition: 

H(f)C(f)A(f)  *  H(f )  -  A(f)  =  0  V-  f  e  (-“,»), 


which  is  satisfied  by  the  formula  for  H  given  in  the  theorem.  Furthermore 
using  Parseval's  relation  in  Eq.  (4-55)  yields  Eq.  (4-53)  for  the 
sequence  of  matrices  C^.  Now,  the  formula  of  Eq.  (4-54)  for  the  minimum 
estimation-error  can  be  obtained  from  the  general  formula  of  Eq.  (4-29) 
by  substituting  the  TSR's  for  kxx>  hQ,  k  into  the  latter  to  obtain: 


M  00 


<e  >  =  ll  B«P[AM  -  l  l  IIP?;  .CP^'A**'**]. 
o  L.  L  -rl  r  L  ,L  ,  r-(n-m)  n-m  m 

p,q=l  r=-°°  n,m  p'  ,q' 

This  number  is  the  trace  of  the  zeroth  element  in  the  sequence  of  matrices 


{ B}0{ (A)  -  {H)S{C}S{A}} 

where  Q  denotes  the  composition  of  discrete  convolution  and  matrix 
product.  Taking  the  z-transform  yields 


B(f)[A(f)  -  H(f)C(f)A(fj]  =  B(f)ll(fj. 

Hence,  <eQ>  is  the  trace  of  the  zeroth  element  of  the  inverse  z-transforra 
of  this  matrix  of  functions,  and  is  given  by  Eq.  (4-54). 

QED 

The  optimum  filter  given  in  this  theorem  has  the  same  structure  as 


that  shown  in  Fig.  (4-4),  except  that  the  input  filters  have  transfer 
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functions  given  by 


e*(f)  =  G*(f)**(f)/K  (f) 

p  p  nn 


rather  than  $*,  and  are  therefore  matched  filters  for  dispersed  versions 
of  the  pulses  in  colored  noise  [2]. 


If  we  consider  the  class  of  optimum  filtering  problems  where  the 
channel  consists  of  nothing  more  than  additive  white  noise  with  PSD  Nq, 
then  its  easily  shown  that  the  formulas  of  Theorems  (4-5),  (4-6),  (4-7) 
for  the  optimum  impulse-response  function  all  reduce  to  the  simplified 
versions: 

h  (t,T)  =  f  I  HP?  ♦  (t-nTH*(T-mT)  (4-56) 

p,q=l  n,m=-°°  p  * 

where  the  z-transforms  of  the  matrix  of  sequences  {H^}  is  given  by  the 
formula 


H(f)  =  A(f)  [E(f)A(f )  +  fM]'1  (4-57) 

where  £  is  the  z-transform  of  the  matrix  of  sequences 
00  00 

E*q  =  /  ♦;(t-rT)*a(t)dt  =  /  **(f)*  (f)ej2lTrTfdf . 

Also,  the  formulas  for  the  time-averaged  value  of  the  minimum  estimation- 
error  given  in  the  previous  three  theorems  reduce  to  the  formula 

1/2T 

<e  >  =  N  /  trace [E(f)A(f)>[E(f)A(f)  +  N  IJ_1]df.  (4-58) 

o  o_1/2T  o 
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Note  the  parallel  to  the  formulas  for  the  optimum  time-invariant  filter 


transfer  function  H  and  for  the  resultant  minimum  value  of  the  time- 
o 

averaged  estimation-error  <eQ>: 


Ho(f) 


Y  *• (f)A(f)**(f) 

NQ  +  Y  *'( f)A(f)$*(f) 


*  “  f  i’(£)A(f)**(f) 

<e  >  =  N  /  — - t - 

0  °-«  N  +  i  4>'(f)A(f)4*(f) 


df  =  N  h  (0) . 
o  o 


(4-59) 


(4-60a) 


These  latter  two  formulas  result  from  substituting  Eq.  (3-7)  for  ft . 

XX 

into  the  general  formulas  of  Eqs.  (4-26),  (4-27). 


Furthermore,  if  the  {^(t-nT)}  are  doubly  orthogonal,  in  the  sense 


/  <p  (t-nT)<j>*(t-mT)dt  =  66, 

P  q  nm  pq* 

as  in  Theorem  (4-5),  then  E(f)  =  1,  and  the  formula  of  Eq.  (4-57)  reduces 
to 

H(f)  =  A(f)[A(f)  +  NqI_]  * 1  (4-61a) 

and  H(f)  is  the  transfer  function  for  a  matrix  sampled-data  Weiner  filter 
for  jointly  WSS  sequences  in  additive  white  noise.  For  this  case  the 
formula  for  the  minimum  error  also  reduces  to 

1/2T 

<e  >  =  N  /  trace  [A  (f )  [A  (f)+N  I_]-1]df  =  N  trace[ljL].  (4-62a) 
0  °-l/2T  0  ~ 

I 

As  discussed  in  Subsection  2f  of  Chapter  III,  there  are  various  CS 
signals  of  interest  whose  TSR  correlation  matrices  are  diagonal.  For 
example,  if  the  autocorrelation  function  satisfies  the  condition 
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0 

0 

0 

L’ 


u 


u 


I 

L.- 


L 


kxx(t+rT^)  - 


e  [0,T], 


then  from  Theorem  (3-3)  of  Chapter  III  the -correlation  matrix  A(f)  for 
the  Karhunen-Loeve  TSR  is  diagonal: 

A  (f)  =  A»S  l  a  e-j2'rTf. 
pq  o  pq  ‘  r 

When  A(f)  is  diagonal,  the  transfer  function  for  the  matrix  of  sample-data 
filters  given  by  Eq.  (4-61a)  reduces  to 


6  A  (f) 

H  (f)  =  . Efl.PP~._- 

Pq  N  +  A  (f) 
o  pp*- 


(4-61b) 


and  is  diagonal.  Hence,  the  optimum  filter  structures  shown  in  Figs. 
(4-4a),  (4-4b)  consist  of  a  parallel  connection  of  independent  paths, 
as  shown  in  Fig.  (4-4c) . 

Also,  when  A(f)  is  diagonal  the  formulas  for  minimum  estimation-error 

given  by  Eqs.  (4-60a),  (4-62a)  reduce  to 

"ojA  (f)i$  (f)|2 
T  p=l  PP  " 


<e  > 


/ 


M 


df 


(4-60b) 


n  y  a  (f)i*  (f)  | 2 
0  T  q=i  w  q 


M  1/2".  N  A  (f) 

<e>=  l  /  -***— 


df. 


p=l  -1/2T  N  +  A  (f) 
r  o  ppv  J 


(4-62b) 


Furthermore,  as  discussed  in  Subsection  2f  of  Chapter  III,  there 
are  various  CS  signals  of  interest  whose  TSR  correlation  matrices  are  not 
only  diagonal,  but  also  constant.  If  we  consider  estimating  one  of  these 
signals  in  additive  white  noise,  then  the  solutions  provided  by  the 
preceding  three  theorems  take  the  particularly  simple  form: 
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•  M 
h„Ct,t)  =  i  i 


app 

o 


«.  -  pp  ♦p(t-nT)**(x-nT) 

n=-“  p=l  N  +  A™  P  p 

*  rt  A 


<e 


.  M  N  APP 

c>  =  T  I  °° 


p-I  N  ♦  APP 
o  o 


(4-63) 


where  M  is  infinite  in  Theorem  (4-5),  and  may  be  finite  in  Theorems 
(4-6),  (4-7). 


Now-  if  M  is  infinite,  then  <e  >  can  be  difficult  to  evaluate  to 

o 

within  a  known  accuracy.  With  this  in  view,  we  present  the  following 
discrete  version  of  Theorem  (4-3): 


THEOREM (4- 8) :  The  time-averaged  value  of  the  minimum  mean-squared 
estimation-error  which  results  from  the  optimum  time-varying  filter  for 
a  CS  signal  in  additive  white  noise  is  contained  within  the  ±1.5  dB 
bounds: 

B/2  <  <eQ>  <  B;  B  =  £  (0)  -  i  \  (APP-Nq)  (4-64) 

P=1 

provided  that  the  correlation  matrix  of  the  TSR  (with  doubly  orthonormal 
basis  functions)  for  the  signal  is  a  constant  diagonal  matrix  whose 
diagonal  elements  dominate  the  noise  PSD  for  all  indices  less  than  some 

integer  Q,  and  vice-versa  for  all  indices  greater  than  Q: 
j  >  No,  P  <  Q 
APP  " 

°  {  <  p  >  Q. 

The  proof  of  this  theorem  is  simply  a  discrete  version  of  that  given 
fcr  Theorem  (4-3).  Recall  that  Theorem  (3-3)  of  Chapter  III  gives  the 
conditions  under  which  the  correlation  matrix  A(f)  for  a  CS  process  will 
be  constant  and  diagonal,  and  that  several  examples  arc  given.  We  will 
discuss  these  and  other  examples  in  the  last  section  of  this  chapter. 


249 


c)  Solutions  based  on  the  harmonic  series  representation, 
i)  Estimation  of  cyclostationary  processes  on  an  infinite  .interval. 

As  brought  out  in  Section  3  of  Chapter  III,  the  harmonic  series 
representation  (HSR)  for  CS  processes  results  directly  from  a  specific 
TSR.  Hence,  the  solution  method  of  Theorem  (4-5)  must  have  a  counterpart 
which  employs  the  HSR.  The  solution  resulting  from  this  counterpart  is 
stated  in  the  following  theorem: 

THE0REM(4-9) :  The  impulse-response  function  for  the  optimum  time-varying 
filter  for  the  general  LMS  estimation  problem  where  the  transmitted  and 
received  signals  x,y  are  jointly  T-CS  and  where  y  is  composed  of  a 
colored  component  w  and  an  uncorrelated  white  component  with  PSD  A  has 
the  representation: 


h  (t,r)  =  l  h  (t-r)e;’27r^pt'qT-)//T  V-  t,t  e  (-»,»)  (4-65) 

u  p,q=-“  Iq 

where  the  matrix  of  Fourier  transforms  of  the  elements  (h  (•)) 

pq 

is  given  by 

i  (5(f)  [D(f)  +  \IJ  _1  ¥■  f  e  [-1/2T,  1/2T] 

H(f)  =  !  (4-66) 

:  0  ¥•  f  t  [-1/2T, 1/2T] 

where  £,  G  are  the  Fourier  transforms  of  the  bandlimited  (to  [-1/2T, 1/2T] ) 

matrices  k,  £  whose  elements  are  given  by 

d  (t)  =  //  v(T-o)v(y)k  (a,Y)e‘;’2lT(pa'qY)/TdodY 

_  00 


g  (t)  =  //  v(T-a)v(y)kxw(o,Y)e":’2TT('pa"qY-)//TdadY 

FH  co 


A  sinQrt/T) 

y  ‘  ~  TTt 


(4-67) 


.  I 


and  are  the  correlation  matrices ■ corresponding  to  thd  HSR's,  for  x,w: 
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d  (t-s)  =  E{d.  (t)dVs)} 
Pm  P  m  '  f 

gpq(t>s)  f  E{4p(tid*Cs)> 


where 


!  1 


x(t)  =  l  a  (t)ej2Tt/T 

P  P‘  , 

,  w(t)  =  l  d  (t)ej2,rpt/T.  (4-68) 

■  p  ,  •  ■  •  , 

I 

Furthermore,  the  time-averaged  value  of  the  minimum  mean- squared 

*  '  > 

estimation -error  is  ,given:by  the  formula: 

1/2T 

<e  >  =  ■  /  trace  [J((f)  -  G(f)  [D(f)+XI]-1G*'  (f)]df  (4-69) 

0  -1/2T  i  1  , 

'  ■  ,  -  '  ' 

where  K(f)  is  the:  Fourier  transform  of  the  HSR  correlation  matrix  for 

i 

x  (defined  as  in  Eq.  (4-67)  for  w). 


The  proof  of  this  theorem  follows1  directly  from  Theorem  (4-5), 

as  indicated  in  Section  3e  of  Chapter  II 1^  by  using  Eq.  (3-88)  which 

5 

relates  the  correlation  matrices  for  TSR's  and  HSR's. 

•  v  t  j 

As  discussed  in 'Section  3e  of  Chapter  III,  a  linear  system  whose 
imDulse-response  function  has  an  Mth  order  HSR,  as  does  hQ  in  this, 
theorem  (with  M  =  ») ,  can  be  realized  with  the  structure  shown  in  Fig.  (3-12) 
and  reproduced  here  as  Fig.  :(4-5).  Although  the  number  i (2M+l)t  of  paths  i 
in  this  optimum  filter  will,  in  general,  be  infinite  there  is  an 

I  ! 

interesting  class  of.  problems  for  which  it  is  finite.  As  discussed  in 
Subsections  3c,d,e  of  Chapter  III,  T-ds  processes  which  are  bandlimited 
to  any  interval,  say  [ -B ,'B]  admit  HSR's  of  order  M  <  BT..  Hence,  the 


i  i 


I 
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optimum  filters  for  such  processes  will  be  finite  HSR  structures. 

The  HSR  structure  for  optimum  filters  has  the  same  interpretation 

as  the  TSR  structure  of  Fig.  (4-4):  The  filter  decomposes  the  received 

signal  into  jointly  WSS  bandlimited  processes,  subjects  these  to  a 

matrix  of  optimum  time-invariant  filters,  and  then  reconstructs  from 

these  filtered  WSS  processes  the  LMS  estimate  of  the  CS  transmitted 

signal  x.  Thus,  by  employing  the  HSR,  we  convert  the  problem  of  LMS 

estimation  of  a  scalar  CS  process  into  the  problem  of  IMS  estimation  of 

a  vector  of  jointly  WSS  (bandlimited)  processes. 

As  already  discussed  in  Section  3e  of  Chapter  III,  the  matrix  of 

time-invariant  filters  employed  in  the  HSR  structure  have  transfer 

functions  H  (f)  which  are  bandlimited  to  [-1/2T, 1/2T] ,  and  can  therefore 
PQ 

be  exactly  realized  with  a  matrix  of  sampled-data  filters,  with  transfer 
functions 


00 


H  (f)  =  l  H  (f-n/T), 

p<r  nL»  p<t 

flanked  at  the  input  and  output,  respectively,  by  T-periodic  impulse 
samplers,  and  ideal  low-pass  filters  (transfer  functions  V (f ) ) . 

There  is,  of  course,  an  equivalent  realization  of  this  HSR  structure 
: which  employs  real  sinusoidal  multiplier  functions  rather  than  complex 
exponentials  (see,  for  example,  Section  2c  of  Chapter  III). 

Paralleling  the  results  obtained  using  TSR's,  we  have  the  following 
simplification  of  the  formulas  in  Theorem  (4-8):  If  the  channel  consists 
of  nothing  more  than  additive  white  noise  with  PSD  Nq,  then  the  matrix 
solution  of  l:q.  (4-66)  reduces  to 


H(f)  =  K(f)[K(f)  +  N  I]"1, 


(4-70) 


which  is  the  transfer  function  for  a  matrix  Weiner  filter  for  jointly 
WSS  processes  in  additive  white  noise.  Also,  the  formula  of  Eq.  (4-69) 
for  the  time-averaged  minimum  mean-squared  estimation-error  reduces  to 


1/2T 

<e>  =  N  /  trace [K(f)[K(f)+N  I]_1]df 
0  0  -1/2T  -  -  o- 


(4-71) 


and  parallels  the  formula  (from  Eq.  (4-27))  for  the  minimum  time-averaged 


error  resulting  from  the  optimum  time-invariant  filter; 


.  IVf~p/T) 

<e  >  =  N  I  - 

°  °-  No  * 


df  =  N  h  (0) , 
o  o  ' 


(4-72) 


(where  we  have  used  Eq.  (3-89)  for  ]!!  . ) 

It  is  interesting  to  note  that  the  optimum  time-invariant  filter 
depends  only  on  the  autocorrelations  of  the  HSR  representors  (the  diagonal 
elements  of  the  correlation  matrix) ,  and  is  independent  of  the  cross¬ 
correlations.  Thus,  the  difference  between  the  optimum  time-varying  and 
time-invariant  filters  can  be  interpreted  in  terms  of  the  fact  the  latter 
ignores  the  crosscorrelations,  whereas  the  former  takes  full  account  of 
them. 


ii)  Estimation  of  periodic  processes  on  a  finite  interval.  The  HSR  is 
particularly  convenient  for  representation  of  periodic  processes;  i.e. 
those  whose  realizations  are  periodic  waveforms.  Thus  the  solution  of 
the  IMS  estimation  problem  for  these  processes  is  particularly  simple 
when  the  HSR  is  employed.  Although  periodic  processes  are  cyclostationary 


processes,  we  can  not  simply  apply  the  results  of  Theorem  (4-9)  to  obtain 
a  LMS  estimate  for  a  periodic  process  since  the  original  equation  for 
optimality  (Eq.  (4-12)),  to  which  this  theorem  gives  a  solution,  is  not 
well  defined  for  periodic  processes;  i.e.,  the  integral  does  not  exist 
(does  not  converge).  This  is  a  result  of  the  fact  that  the  optimum  filter 
is  not  BIBO  stable,  and  is  not  therefore  a  permissable  filter  (see 
Section  2  of  this  chapter) .  The  difficulty  can  also  be  relegated  to  the 
fact  that  the  problem  of  estimation  of  a  periodic  process  on  an  infinite 
interval  is  a  singular  estimation  problem  and  will  result  in  zero  mean- 
squared  estimation-error. 

Now,  a  more  meaningful  problem  is  that  of  noncausal  LMS  estimation 

of  a  periodic  process  on  a  finite  interval.  This  estimation  problem  (the 

fixed-interval  smoothing  problem)  is  non-singular  and  is  characterized 

by  the  optimality  condition: 

T 

f 

/  ho(t,o)kyy(f,o)do  =  kyx(r,t)  Vt.x  e  LT.,Tf]  (4-73) 

i 

which  follows  directly  from  Eq.  (4-7)  with  T  =  T  =  |t.-TJ  <  °°. 

For  this  problem,  we  have  the  following  general  solution: 

THE0REM(4-10) :  The  impulse-response  function  for  the  optimum  fixed- 
interval  smoother  for  a  periodic  process  in  additive  white  noise  with 
PSD  Nq  has  the  representation: 

h  (t,x)  =  l  h  ej2'(ptiT)/T  V-  t.x  e  [-NT/2,NT/2] 

p,q 

where  the  matrix  of  harmonic  coefficients  is  given  by 

H  =  K  [NT  K  +  N^lJ  '  1  (4-74) 


where  the  elements  of  the  matrix  J(  are  given  by 
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T/2 


k  .L  II  k  (t.Oe-J^-l^dtdt. 
pq  "j2  xxv 

so  that  K  is  the  correlation  matrix  for  the  HSR  representors: 


(4-75) 


k  =  E{a  a*} 

pq  P  q 


where 


j2irpt/T 


x(t)  =  l  a  e 
P  F 

(see  Sec.  3d  of  Chapter  III).  Furthermore,  the  time-averaged  value  of 
the  minimum  mean-squared  estimation-error  is  given  by  the  formula 


N  N 

<e  >  =  —  trace[K[K  +  —  I]"1]. 


(4-76) 


NT  NT 

The  proof  of  this  theorem  can  be  obtained  simply  by  substituting 
the  representations  for  hQ  and  k  ,  into  the  reduced  version  of  Eq.  (4-73): 
NT/ 2 

J  h  (t,o)k  (T,0)do  +  N  h(t,T)  =  k  (t,x),  V-t.T  e  [-NT/2.NT/2] . 

2  »'  AA  U  AA 

For  comparison,  we  present  tire  following  general  solution  for  the 
optimum  time-invariant  fixed-interval  smoother: 


THEOREM (4- 1 1) :  The  impulse-response  function  for  the  optimum  time-invariant 
fixed-interval  smoother  for  a  periodic  process  in  additive  white  noise 
with  PSD  N  has  the  representation: 


ft  (t-r)  =  l  h  e32uq(t-T)/T>  Vt,T  e  [-NT/ 2, NT/ 2] 

q  q 

where 


k 

h  =  - -4 -  (4-77) 

q  NTk  +  N 

-q  0 


2SS 


L 


L 


and  where  {k^}  are  the  Fourier  series  coefficients  for  the  periodic 
autocorrelation  function  for  the  stationarized  version  of  the  periodic 
process,  and  are  identical  to  the  diagonal  elements  of  the  HSR 
correlation  matrix: 


L. 


L 


k  =  k  . 

q  qq 

Furthermore,  the  minimum  value  of  the  time-averaged  estimation  error 
is  given  by  the  formula: 


N 


.  0  V 

<e  >  =  —  ) 

r\  « 


(4-78) 


NT  q  k  +  N  /NT 
^  -q  o 


The  proof  of  this  theorem,  like  that  for  the  preceding  theorem, 


can  be  obtained  simply  by  substituting  Eq.  (4-77)  for  hQ,  and  the  Fourier 
series  representation  for  k  ,  into  the  optimality  condition: 

XX 


NT/ 2 

/  ho(t-a)'kVxx(T-o)do+Nnhn(t-T)  =  kVvy(t-T)  n,i  e  [-NT/2.NT/2] . 
-NT/2 


xx' 


Note  that  as  the  length  of  the  observation  interval  approaches 
infinity,  the  estimation  errors  of  Theorems  (4-10),  (4-11)  both  approach 
zero,  as  expected  from  the  preliminary  discussion. 

The  optimum  smoothers  of  Theorems  (4-10),  (4-11)  can  be  realized 
with  the  structure  shown  in  Fig.  (4-6),  where  M  is  the  highest  non-zero 
harmonic  in  the  signal  being  smoothed.  The  numbers  {a^}  at  the  outputs 
of  the  matrix  of  attenuators  are  the  LMS  linear  estimates  of  the  Fourier 
series  coefficients  of  the  periodic  signal  x.  For  the  time-invariant 
smoother  of  Theorem  (4-11),  the  matrix  of  attenuators  is  diagonal  (h  =  h^) 
so  that  the  overall  structure  becomes  a  parallel  connection  of  2M+1  paths. 


(4-79) 


From  this  it  can  be  seen  that  the  difference  between  the  optimum  time- 
varying  and  time-invariant  smoothers  is  that  the  latter  ignores  the 
crosscorrelation  between  various  harmonics. 


d)  Solutions  based  on  the  Fourier  series  representation  for 
correlation  functions.  We  begin  with  a  theorem: 

THEOREM (4- 12 ) :  The  impulse-response  function  for  the  optimum  filter 
for  the  general  LMS  estimation  problem  where  the  transmitted  and  received 
signals  x,  y  are  jointly  T-CS  and  where  y  is  composed  of  a  colored  component 
w  and  an  uncorrelated  white  component  with  PSD  X  has  the  representation: 

00 

ho(t,x)  =  l  hn(t-r)e-’7m(-t+T^  T  V- t,x  e  (-“,<»)  (4-80) 

u  n=-“ 

where  the  Fourier  transforms  {Hn>  of  the  time-invariant  impulse-response 
function  {h^}  are  the  solutions  of  the  linear  algebraic  equations: 

00 

l  Cnm(f-(n+m)/2T)Hm(f-m/2T)  +  AHn(f-n/2T)  =  Gn(f-n/2T)  H  (4-81) 
m=-°° 

where  the  (C  )  and  (G  )  are  the  Fourier  transforms  of  the  coefficient - 
n  n 

functions  for  the  FSR's  for  the  correlation  functions  k  ,  k  : 

ww  xw 

-  T  T^/2k»(s*t/2-s't/2)e'j2’!’S/Tds 

T/2 

g  (t)  =  i  /  k  (sn/2,s-t/2)e'j2™s/Tds  (4-82) 

n  *  »p  j  ^ 

(see  Section  4a  of  Chapter  III). 

The  proof  of  this  theorem  is  outlined  in  terms  of  Eqs.  (3-117)-  (3-121) 
in  Section  4c  of  Chapter  III. 

There  does  not  appear  to  be  any  way  to  obtain  a  general  closed-form 
solution  to  this  infinite  set  of  equations;  however,  if  the  functions 
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{CnJ,  {G^}  satisfy  certain  bandliraiting  constraints,  then  solutions 
are  possible,  as  discussed  in  Section  4c  of  Chapter  III.  For  example: 


THE0REM(4-13) :  The  impulse-response  functions  for  the  optimum  filter 
for  a  T-CS  process  x  in  additive  white  noise  with  PSD  Nq  has  the 


representation : 


Mt.T)  -  l  h  (t-t)eJ’n't")/T 


(4-83) 


where  the  Fourier  transforms  {H  }  of  the  functions  {h  }  are  given  by 

n  n  6 


the  formulas: 


H  (f)  = 


Cn(f)[l  -  Ho(f+n/2T)] 
Nq  ♦  Cfl{f-n/T) 


H0(f)  - 


No  +  R(f) 


(4-84) 


wj~.ere 


|C  (f-n/2T) | 2 
R(f)  -  C  (f)  -  I  — - 

w  l r\  ti  r*  re  —  /^r 


r4°  Nq  +  CQ(f-n/T) 


(4-85) 


provided  that 


Cn(f)  =  0,  |f|  >  1/4T,  ^n/0, 


(4-86) 


where  {C  }  are  the  Fourier  transforms  of  the  FSR  coefficient  functions: 
n 


,(t)  =  i  /  R(s+t/2,s-t/2)ej27Tns/Tds. 
i  *  T/  2 


(4-87) 


Furthermore,  the  time-averaged  value  of  the  minimum  mean-squared  estimation- 
error  is  given  by  the  formula: 


»  N  R(f ) 

<e  >  =  /  — ± - df  . 

0  -»  N  +  R(f) 

o 


(4-88  j 
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Proof:  Using  the  condition  of  Eq.  (4-86)  in  Eq.  (4-81)  of  Theorem  (4-12), 
where  =  Cn,  A  =  Nq,  yields  the  equations: 

l  C_m(f-m/2T)Hm(f-m/2T)  +  (C0(f)+No)H0(f)  =  CQ(f),  (4-89) 

(C0(f-n/T)+No)Hn(f-n/2T)  +  Cn(f-n/2T)H0(f)  =  Cn(f-n/2T)  V-  n  M.  (4-90) 

Now,  Eqs.  (4-84),  (4-85)  follow  directly  from  Eqs.  (4-89),  (4-90). 

Furthermore,  the  formula  of  Eq.  (4-88)  results  from  substituting 
Eqs.  (4-83),  (4-84)  into  the  formula  of  Eq.  (4-32). 

QED 

Note  that  if  k^x  has  an  order  FSR  satisfying  the  constraint  of 
Eq.  (4-86),  then  so  too  does  the  solution  h  provided  by  Theorem  (4-13). 

As  discussed  in  Section  4c  of  Chapter  III,  a  linear  system  whose  impulse- 
response  function  has  an  M  order  FSR  can  be  realized  with  the  (2M+l)-path 
structure  shown  in  Fig.  (3-14a),  or  with  the  simplified  structure  shown  in 
Fig.  (3- 14b)  and  reproduced  here  as  Fig.  (4-7),  where  the  impulse- 

response  functions  for  the  path  filters  with  transfer  functions  G^ 
are  related  to  the  original  coefficient-functions  as  follows: 

gnj(t)  =  2Re{hn(t)}cos(Trnt/T)  -  2Im{hn(t)  }sin(7rnt/T) 

gn2(t)  =  -2Re{hn(t) }sin(mit/T)  -  2Im{hn(t) }cos (tmt/T) .  (4-91) 

Note  that,  as  mentioned  in  subsection  4c  of  Chapter  III,  the 
solution  method  of  Theorem  (4-13)  can  be  extended  to  the  case  where  the 
Fourier  coefficient  functions  satisfy  the  less  restrictive  constraint 

Cn(f)  =  0,  |f  |  >  M/4T,  V-  n  /  0. 
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However,  the  method  then  requires 
of  functions,  and  does  not  appear 
solution  method  based  on  the  HSR, 


inversion  of  a  (2M-1)  x  (2M-1)  matrix 
to  offer  any  advantage  over  the 
which  is  considerably  less  cumbersome. 


e)  Solutions  based  on  inverse-operator  and  feedback-system  methods, 
i)  Introduction.  The  basic  integral  equation  derived  in  the  second 
section  of  this  chapter: 

oo 

/  h  (t,s)k  (s,x)ds  =  k  (t,T)  V- t,r  e  (-“,“),  (4-92) 

-00  ''  * 

which  implicitly  specifies  the  impulse-response  function  for  the 
optimum  filter,  can  be  interpreted  as  an  operator  equation.  Specifically, 
if  we  denote  the  linear  integral  transformations  (operators)  with 
kernels  k  (*,*)»  h  (*,*)»  k  (*,♦)  as  K  ,  H  ,  K  ,  then  the  operator 
counterpart  of  this  integral  equation  can  be  written  as 


H  *K  =  K  . 

_q  -Zyy  -Xy 


This  equation  has  the  formal  solution 


H  =  K  «K  -1 
-o  — xy  -yy 


where  JC*  is  the  inverse  operator  corresponding  to  K^: 
K  -K"1  =  I, 

-yy  -yy  - 


(4-93) 


(4-94) 


(4-95) 


where  I_  is  the  identity  operator. 

Now,  if  we  denote  the  kernel 
then  the  impulse-response  function  for  the  optimum  filter  is  formally 


Now,  if  we  denote  the  kernel  for  this  inverse  operator  as  k’y(*,*)> 
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given  by 


h  (t,x)  =  /  k  (t,o)k"l(a,T)d0. 

_ oo  *  y  y 


(4-96) 


Furthermore,  this  optimum  filter  can  be  realized,  as  shown  in  Fig.  (4-8), 
as  the  cascade  connection  of  the  two  linear  systems  which  are  characterized 
by  the  operators  K-^  ,  K  ,  and  have  impulse-response  functions 
k' 1  (*,•)>  k  (*,*). 

yy  v  7  '  7  ^y  v  9  * 

There  are  a  number  of  interesting  LMS  estimation  problems  for  T-CS 
processes  where  the  kernels  of  the  operators  K^,  can  be  decomposed 
in  such  a  manner  that  the  system  diagram  of  Fig.  (4-8)  can  be  manipulated 
into  the  feedback  configuration  shown  in  Fig.  (4-9),  where  is  a  known 
time- invariant  system,  p(*)  is  a  known  T-periodic  function,  and 
are  known  periodically  (T;  time-varying  systems.  Furthermore,  there  is  a 
non-tiivial  set  of  circumstances  under  which  the  impulse-response  function 
hQ  for  this  system  can  be  obtained  explicitly: 

Namely, 


K  (f)  =0,  | | f | -  M/2T |  >  N/2T 


P  =0. 


|  n  |  -  M  |  <  N ,  ¥-n^0 


(4-97) 


where  N,  M  are  nonnegative  integers,  K  (»)  is  the  transfer  function  for 

K  ,  and  {p  }  are  the  Fourier  coefficients  for  p(*)> 

-O  n 

For  M  >  1,  K  is  a  bandpass  filter,  and  for  M  =  0,  K  is  a  lowpass 
“0 

filter.  For  N  =  1,  M  =  0,  p(’)  is  unconstrained,  and  for  N  >  1,  M  =  0, 
p(»)  contains  only  frequency-components  which  are  two  high  to  be  passed 
by  the  filter  K^,  and  for  N  =  1 ,  M  ■>  1,  p(*l  contains  frequency  components 
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which  are  too  low  to  be  passed  by  Kg.  Now,  when  the  conditions  of 
Eq.  (4-97)  are  satisfied,  the  system  of  Fig.  (4-9)  is  unaffected  by 
the  replacement  of  the  periodic  multiplier  in  the  feedback  path  with  a 
constant  attenuator  whose  attenuation  is  pg,  the  average  value  of  p(*)- 
Thus,  the  entire  feedback  loop  is  time- invariant,  and  has  transfer 


function  given  by 

(1/p  )K  (f) 

K -(f)  =  - — -  ,  (4-98) 

(l/p0)  +  Kg(f) 

so  that  the  impulse-response  function  for  the  entire  system  of  Fig.  (4-9)  is 


(4-98) 


00 

ho(t,x)  =  //  k2(t:o)k3(a,s)k1(s,Tjdsdo,  (4-99) 

-00 


where  k^  is  the  inverse  Fourier  transform  of  K^. 

We  refer  to  this  method  of  solving  for  optimum  time-varying  filters 
as  the  inverse-operator-feedback-system  (IOFS)  method.  We  now  present 
four  classes  of  practical  optimum-filtering  problems  for  which  the 
general  solution  of  Eqs.  (4-98),  (4-99)  is  valid.  In  the  next  section, 
we  illustrate  these  classes  of  problems  with  specific  problems. 


ii)  Examples : 

1)  WSS  signal  subjected  to  time-invariant  dispersion,  periodic 
attenuation,  and  additive  WSS  white  noise:  For  the  class  of  optimum¬ 
filtering  problems  where  a  WSS  signal  is  transmitted  through  a  channel 
consisting  of  a  time-invariant  dispersion  with  impulse-response  function 
g,  T-periodic  attenuation  q,  and  additive  WSS  white  noise  with  PSD  Nq, 
the  correlation  functions  in  Eq.  (4-92)  take  the  form: 
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kyy(t»s)  =  //  txx(?-Y)g{t-a)g(s-Y)dodyq(t)q(s)  ♦  NQ«(t-s) 

k  (t,*)  =  /  k  (t-o)g (s-a)dcq (s) ,  (4-100) 

"  _«•  ** 

50  that  the  systems,  with  these  functions  as  impulse  responses,  take  the 
foras  shown  ir  Tig.  (4-10).  Now,  using  eleoentary  algebra  for  linear 
system  f36),  it  is  easily  shown  that  the  systes  of  Fig.  (4-8)  is 
eqai valent  to  that  of  rig.  (4-9),  where  the  impulse-response  functions 

are  given  by 


kj(t,s)  =  q(t)S(t-s) 

Kfir)  --  r*i£xx(£)\G( f)l'}/So 
k7i  z)  =  r*l{l/G(f*i, 


(4-101) 


2nd  where  p(t)  =  o2(t).  2iov  if  tee  channel  dispersion  or  the  signal 
P52  ar^  bandpass,  2nd  the  periodic  fluctuations  in  attenuation  are 
relatively  low- frequency,  or  if  the  dispersion  or  PSD  are  low-pass,  and 
the  fluctunti ms  in  attenuation  arc  relatively  high-frequency,  (so  that 
the  conditions  of  Eu.  (4-S?)  a^e  satisfied s.  then  the  general  solutions 
c:  Sqs.  ,.4-99),  ror  the  optioua  filter,  sre  valid. 

(2  j  CS  signer  in  additive  KS5  white  noise;  For  the  class  of  optiaun- 
fiilering  problems  where  c  i.5  signal,  which  is  product  of  a 
deterai r.i Stic  T-periodic  function  q.  and  a  ifSS  process  ?■,  :  s  transmitted 
through  a  ccannc ;  t assisting  or.!)'  of  add; live  KSf  white  noise  vith  PSD 
11  ,  the  correlation  functions  in  Eq  (4-9;)  lake  t.»e  corns- 


5.  U.»«  =  q{?  ik  (t-s)q>)  ♦  .  6(1  ) 

yy  zz  o 

k  (t,s)  •  q(t)k  (r-s)u(s;, 

£  v  ZZ 


'4-102* 
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so  that  the  systems,  with  these  functions  as  impulse-responses,  take 
the  forms  shown  in  Fig.  (4-11).  Using  the  algebra  for  linear  systems, 
it  is  easily  shown  that  the  system  of  Fig.  (4-8)  is  equivalent  to  that 
in  Fig.  (4-9),  where  the  impulse-response  functions  are  given  by 

kj(t,s)  =  q(t)6(t-s) 

krt(r)  =  k  (t)/N 
0  22  o 

k2(t,s)  =  q(t)6(t-s) ,  (4-105) 

and  where  p(t)  =  q2(t). 

Now  if  the  "baseband"  signal  z  is  lowpass  and  the  periodic 
fluctuations  in  the  "carrier"  are  relatively  high-frequency,  or  if  the 
process  z  is  bandpass,  and  the  multiplicative  factor  q  is  relatively 
slow-varying  (so  that  the  conditions  of  Eq.  (4-97)  are  satisfied),  then 
the  general  solution  of  Eqs.  (4-98),  (4-99)  for  the  optimum  filter,  are 
valid. 

(5)  WSS  signal  in  additive  CS  white  noise:  For  the  class  of  optimum- 
filtering  problems  where  a  WSS  signal  is  transmitted  through  a  channel 
consisting  of  only  additive  CS  white  noise  with  autocorrelation  function 
q2(r)S(t-s),  the  correlation  functions  in  Eq.  (4-92)  take  the  forms: 

k  (t,s)  =  kxx(t-s)  +  q2(t)6(t-s) 

kxy(t,s)  =  kxx(t-s),  (4-104) 

so  that  the  systems,  with  these  functions  as  impulse  responses,  take  the 
forms  shown  in  Fig.  (4-12).  Using  the  algebra  of  linear  systems,  it  is 


1 


?64 


>  * 

again  easily  shown  that  the  systea  of  Fig.  (4-3)  is  equivalent  to  that 

>  •  > 

in  Fig.  (4-9),  where  the  iapulse-response  functions-  are  given  by  , 

*  i 

kj(t,s)  *  q'2(t)«(t-s)  , 

*  • 

I 

i 

krt(x)  =  k  (t)  ' 

0  xxv 

i 

k2(t,s)  =  i(t-s),  •  ,  (4-105) 

and  where  p(t)  =  q"2(t). 

Now  if  the  periodic  fluctuations  in  the  intensity  of  the  additive 

«  •  «  . 

white  noise  are  either  low-frequency  compared  to  the  bandpass  signal,  ’ 

or  are  high-frequency  compared  to  the  lowpass  signal  (so  that  Eq.  (4-97) ' 
is  satisfied),  then  the  general  solution  of  Eqs.  (4-98),  (4-99) ,  for  the 
optiaua  filter,  are  valid.. 

i  • 

(4)  CS  signal  (obtained  froa  periodically  tiae-scaled  NSS  process)  :in 

1  '*  •  :  t 

additive  NSS  white  noise:  For  the  class  of  optiaua-fxltering  probleas 

where  a  CS  signal,  which  is  obtained  froa  a  NSS  process  z  via  a  periodic 

tiae-scale  transformation  g(t)  -  t  +  q(t)  (as  discussed  in  Section  2c  of 

Chapter  II),  is  transmitted  through  a  channel  consisting =only  of  additive 

NSS  white  noise  with  PSD  N  ,  the  correlation  functions  in  Eq.  (4-92) 

o 

take  the  forms: 

kyyft.sj  =  kz2(g(t)-g(s))  +  NQ6(t-s) 


k  (t,s)  =  k  (g(t)-g(sj), 
xy  +z 


(4-106) 


so  that  the  systems,  with  these  functions  as  impulse  responses,  take 
the  forms  shown  in  Fig.  (4-13).  Using  the  algebra  of  linear  systems, 
it  is  easily  shovm  that  the  system  of  Fig.  (4-8)  is  equivalent  to 
that  of  Fig.  (4-9),  where  the  impulse-response  functions  are  given  by 


kj(t,s) 

VT)  = 

k2(t,s) 


=  <$(t-g(s)) 


k 

zz 


*  5(g(t)-s) 


I 


(4-107) 


and  where  p(t)  =  l/g'(t).  where  the  prime  indicates  differentiation. 

Now  if  the  periodic  fluctuations  in  the  time-scale  transformation 
(i.e.,  in  l/(l+q'(t)))  are  either  low-frequency  compared  lO  the 
bandpass  WSS  process  z,  or  are  high-frequency  compared  to  the  lowpass 
WSS  process  z  (so  that  Eq.  (4-97)  is  satisfied),  then  the  general 
solution  of  Eqs.  (4-89),  (4-99)  are  valid. 
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6.  Examples  of  Optimum  Filters  and  Improvements  in  Performance  - 

a)  Introduction.  In  this  section  we  present  a  number  of  specific 
examples  of  solutions  for  optimum  filters  for  cyclostationary  processes, 
and  of  evaluations  of  the  performance  index  I  (defined  in  Eq.  (4-28)), 
which  indicates  the  degree  of  improvement  in  the  performance  of  the 
optimum  time-varying  filter  over  that  of  the  optimum  time-invariant 
filter.  We  classify  the  examples  according  to  the  method  of  solution 
employed,  so  that  the  examples  in  each  of  the  subsections  6b-6e  correspond 
to  the  methods  presented  in  each  of  the  subsections  5b-5e. 

For  simplicity,  we  consider  only  those  estimation  problems  where 
the  channel  consists  of  nothing  more  than  additive  white  noise  with 
PSD  No  (unless  otherwise  stated.) 


b)  Examples  employing  translation  series  representations, 
i)  Frequency-shift-keyed  signals.  As  a  specific  example  of  a  synchronous 
M-ary  signal,  we  consider  the  frequency-shift-keyed  signal  (Model  (12) 
of  Chapter  II)  which  takes  the  form 

CO 

x(t)  =  l  w(t-nT)cos(2aanfot),  (4-108) 

n=-“ 

where  the  frequency- factors  {a^}  are  statistically  independent  random 
variables  which  take  on  each  of  the  values  in  the  M-ary  alphabet 
{1,2,...,M}  with  probability  1/M,  and  where  w  is  a  sine-pulse: 


w(t)  =  *23 


sin(sBt) 
C*Bt)  1 


(4-109) 


where  the  single-sided  bandwidth  B  <  fQ,  and  where  is  SGae  positive 
integer.. 
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As  shown  in  Secs.  2e,  2f  of  Chapter  II,  x  admits  an  M1*1  order  TSR 
with  the  orthonormal  basis  functions 


4>p(t)  =  w(t)cos(2?ip.fot) 


and  correlation  matrix  (for  the  centered  process  x  -  m' ) 

X 


A  (f)  =  i  5  -  - 

pq  M  pq  .  M2 


(4-110) 


(4-111) 


Now,  from  Theorem  (4-6),  the  optimum  (minimum  error-variance) ^time- 

varying  filter  for  this  FSK  signal  in  white  noise  has  impulse-response 

function  given  by  Eq.  (4-56)..  and  has  the  realization  shown  in  Fig.  (4-4), 

where  the  transfer  function  for  the  matrix  of  sampled-data  filters  is 

given  by  Eq.  (4-61a) .  The  structure  of  this  optimum  filter  has  an 

interesting  interpretation.  The  bank  of  input  filters  are  matched  to 

each  of  the  M  possible  pulses  that  occur  in  the  signal.  The  periodically 

sampled  outputs  of  these  matched  filters  are  passed  through  a  discrete 

matrix  Wiener  filter,  the  outputs  of  which  can  be  shown  to  be  the  IMS 

estimates  of  the  a-posteriori  probabilities;  i.e.,  the  p**1  output  at 

til 

time  nT  is  the  LMS  estimate  of  the  probability  that  the  n  symbol 
"  | 

transmitted  was  the  p  letter  of  the  M-ary  alphabet,  a  =  p,  given 

* 

* 

that  y  was  received  [62].  These  probability  estimates  are  used  to 
weight  the  pulses  which  are  regenerated  at  the  output  of  the  filter. 

See  Fig.  (4-14). 

The  time-averaged  minimum  estimation-error  is  given  by  Eq.  (4-62a), 
and  reduces  to  the  expression 


28  See  the  last  paragraph  at  the  end  of  section  2  of  this  chapter  for  a 
discussion  of  the  relationship  between  minimum- mean-scuared-error  and 
aininura-error- variance. 
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N 

<e  >  =  =2.  trace  '• .  fA  +  N  I]_1 
o  T  — o ' — o  o 

=  i 

T  MN  +  1 
o 


(4-112) 


(using  Theorem  (3-4)). 

The  optimum  time- invariant  filter  has  transfer  function  given  by 
Eq.  (4-59),  and  the  resultant  minimum  time-averaged  estimation-error 
is  given  by  Eq.  (4-60a): 


«  r  t°  Aopiyf)i2 

<e  >  -  7  /  - - - — 2 - df,  (4-113) 

°  p=l-~  No  APP|«p(f)|2 

where  we  have  used  the  fact  that  4  (f )  $*  (f )  =  1 4>  (f )  1 2& 

p  q^  '  1  pv  pq 

Now,  since  the  {S^J.are  rectangular  with  height  1//2B  and  double-sided 
bandwidth  2B,  then 


<e  >  =  2BMN  - .  (4-114) 

°  °  (M- 1 ) / 2  BTM<  MN 

Using  an  effective  noise-bandwidth  of  2BM  to  obtain  a  noise  variance  of 
2BMNq,  and  using  the  time-averaged  signal-variance  (M-1)/MT,  we  can 
express  the  performance  index  of  Eq.  (4-2S)  in  terms  of  the  signal-to- 
noise  ratio  (of  variances)  0: 

<e  >  1  +  p2BTM/(M-l) 

I  A  _2_  = - .  (4-ll5) 

<e  >  1  +  p 

o 

The  lowest  practical  bandwidth  of  the  envelope  w  corresponds  to 
transmitting  pulses  at  the  Nyquist  rate,  and  is  B  =  1/T.  This  results  in 
a  minimum  improvement  factor  (vaiuc  of  performance  index)  of  6  dP  for 
binary  ESK  (M  =  2)  in  low  noise.  As  the  bandwidth  is  increased  in  order 
to  decrease  intcrsymbol  interference,  the  low  noise  improvement  factor 
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increases  in  direct  proportion,  which  means  that  the  time-varying 
filter  becomes  more  attractive,  relative  to  the  time- invariant  filter, 
as  the  quality  of  transmission  increases. 

It  xs  interesting  to  note  that  for  signal-to-noise  ratios  as  low 
as  unity.,  there  can  still  be  a  substantial  improvement: 


1=1/2+  BTM/ (M-l)  for  p  =  1.  (4-116) 

ii)  Time-division-multiplexed  PAM  signals.  An  interesting  (and  perhaps 

more  practical)  alternative  to  the  time-division-multiplexing  schemes 

TDM. ,  TDM2  discussed  in  Chapter  II  (Models  (3),  (5))  is  the  TDM  of  PAM 

scheme,  which  was  briefly  mentioned  in  Sec.  2f  of  Chapter  III.  For  this 

scheme,  we  consider  M  mutually  uncorrelated  WSS  component  signals  {x^3 

with  autocorrelation  functions  {k  }  and  PSD's  {K  }  which  are  bandlimited 

P  P 

to  xhe  interval  [-1/2T,1/2T] ,  The  signals  are  periodically  sampled  every 
T  seconds  to  amplitude-modulate  orthonormal  pulses  {6(t-pT/M);p  =  1,2, — ,M), 
so  that  the  composite  TDM-PAM  signal  takes  the  form 


M 


x(t)  =  /f/M  l  l  x (nT)*(t-pT/M-nT). 
p=l  n=-°°  ” 

.th 


(4-117) 


Clearly  x  admits  an  M  order  TSR,  and  since  the  {x^}  are  uncorrclated, 
then  the  correlation  matrix  is  diagonal  with  on-diagonal  elements 


Vf)  =  1  I  VnI>',2"’’Tf  =  s  l  y  (4-iis) 

n  v  o  * 

New,  from  Theorem  (4-6),  the  optimum  time-varying  filter  for  this 
TDM  signal  in  white  noise  has  impulse-response  function  given  by  Eq.  (4-56), 
and  has  the  realization  shr.wn  in  Fig.  (4-4),  where  the  transfer  function 
for  the  diagonal  matrix  of  samplcd-data  filters  is  given  by  Eq.  (4-61b). 
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The  structure  of  this  optimum  filter  has  an  interesting  interpretation. 

Since  the  matrix  of  sampled-data  filters  H  is  diagonal,  then  the  overall 
structure  is  simply  the  parallel  connection  of  M  independent  paths  as 
shown  in  Fig.  (4-4c).  The  pt^1  path  consists  of  a  discrete  Wiener  filter, 
for  the  p  sequence  of  amplitudes,  flanked  at  the  input  and  output, 
respectively,  by  a  pulse-amplitude  demodulator  and  a  pulse-amplitude 
modulator.  Hence,  the  optimum  time-varying  filter  demodulates  the  signal, 
optimally  filters  it  at  "baseband",  and  then  reraoaulates  it.  See  Fig.  (4-15). 

The  time-averaged  minimum  estimation-error  is  given  by  Eq.  (4-62b) 
which  reduces  to  the  expression 


<• 


M  1/2T 

;  >  =  N  l  J 


K  Cf) 

-2 -  df. 


p=l  -1/2T  MN  +  K  (f) 
r  o  p  ' 


(4-119) 


The  optimum  time-invariant  filter  has  transfer  function  given  by 
Eq.  (4-59),  and  the  resultant  minimum  time-averaged  error  is  given  by 
Eq.  (4-60b).  If  we  assume  that  the  pulse  <}>  is  the  interpolating  sine- 
pulse  whose  Fourier  transform  is  ideal  low-pass  with  double-sided 


bandwidth  M/T  and  height  /T/M  ,  then  Eq.  (4-60b)  reduces  to 

M 


y  k  (f) 

„  1/21  L.  p 

<e  >  =  N  /  2-=^ - 


-1/2T  MN  +  i  l  K  (f) 
°  M  q=l  q 


df . 


(4-120) 


Now,  if  the  M  processes  ix^}  are  identical,  then  the  TDM  process 
degenerates  to  a  iVSS  process,  and  1=1:  there  is  no  improvement. 

However,  if  the  individual  processes  are  different,  there  can  be  substantial 
improvement.  For  example,  if  the  processes  have  disjoint  PSD’s,  then 
Eq.  (4-120)  reduces  to 


i 
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„  "  M  1/2T  K  (f) 

<e  >  =  N  l  J  - E— ■ 


df. 


(4-121) 


p=l  -1/2T  MN  +  J7  K  (f) 
r  o  M  J 


and  there  is  a  low-noise  improvement  factor  of  I  =  M.  Similarly,  if 

tJl 

all  but  one  (say  the  q  )  of  the  M  baseband  signals  are  zero,  then  the 
formulas  of  Eqs.  (4-119),  (4-120)  for  estimation-error  become  : 


1/2T  K  (f) 

<e  >  =  N  /  - 3 - df 

A  A  4 


-1/2T  MN  +  K  (f) 
o  q 


'V,  , 

<e  >  =  N  J 

A  A 


1/2T 


o  o  1  V/  Kit  .  ^  tr 


-1/2T  MNo  +  ^  Kq(f) 


df. 


(4-122) 


(4-123) 


and  there  is  again  a  low-noise  improvement  factor  of  I  =  M. 
iii)  Frequency-division-multiplexed  signals .  A  multiplexing  scheme 
which  is  in  some  ways  a  frequency-time  dual  of  TDM  schemes  is  the 
frequency-division-multiplexing  (FDM)  scheme  discussed  in  Chapter  II 
(Model  (4)).  We  consider  here  FDM  signals  of  the  form 


M 


x(t.)  =  l  x  (t)cos(MQt  +  27ipt/TQ) 
P=1  p 


(4-124) 


where  wo  is  an  integer-multiple  of  2a/T,  and  T  is  an  integer-multiple  of 
of  Tq,  and  where  the  M  component  signals  {x^J  with  correlation  functions 
(k  J  are  assumed  KSS,  uncorrelated,  and  bandlimited  to  [-1/2T,1/2T] . 

From  the  sampling  theorem  (Theorem  (2-2)),  the  component  signals  admit 
the  representations 


X„(t)=  l  x  (nT)  ?i-1°tt-nT)/T 


(4-125) 


n 


» (t-nT)/T 


th 


so  that  the  composite  FDM  signal  admits  the  M  order  TSR 

M  00 

x(t)  =  Sm  i  l  x  (nT)4  (t-nT), 
p=l  11=—  1  1 


(4-126) 


where  the  basis  functions  are  given  by 


4  (t)  =  fijl  cos  {w  t  ♦  2*pt/T  )  Sin(fft^  .  (4-127) 

P  o  o  (ift/T) 

These  basis  functions  are  doubly  orthonormal ,  and  the  TSR  correlation 
matrix  A(f)  is  diagonal  with  on-diagonal  elements  given  by 

A  (f)  =  \  l  k  (nT)e_j2,mTf  =  |  l  K  (f+m/T).  (4-128) 

Pr  n  "  *  m  p 

The  duality  with  the  TDM-PAM  scheme  is  now  evident  since  the 
correlation  matrices  are  identical  (except  for  a  scale  factor) ,  and  the 
basis  functions  are  time- frequency  duals  (if  the  pulse-shape  used  in  the 
TDM  scheme  is  the  interpolating  sine-pulse)  in  the  sense  that  the  M  FDM 
basis  functions  are  frequency-translated  sine-pulses,  and  the  M  TDM 
basis  functions  are  time-translated  sine-pulses. 

Now,  from  Theorem  (4-6),  the  optimum  impulse-response  function  is 
given  by  Eq.  (4-56),  and  has  the  realization  shown  in  Fig.  (4-4c),  where 
the  transfer  functions  for  the  diagonal  matrix  of  sampled-data  filters 
are  given  by  Eq.  (4-61b).  As  expected,  the  structure  of  this  optimum 
filter  parallels  that  for  the  TDM-PAM  signal.  As  shown  in  Fig.  (4-16), 
it  consists  of  M  independent  paths.  The  p1*1  path  contains  an  ideal  bandpass 
input  filter  'which  passes  only  the  p**1  component  signal  Xp)  followed  by 
a  periodic  sampler,  and  a  discrete  Wiener  filter,  and  an  ideal  bandpass 
output  filter,  which  reconstructs  a  smooth  signal  from  the  filtered  samples. 

The  tine-averaged  minimum  estimation-error  is  given  by  Eq.  (4-62b) 


which  reduces  to 
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<e  >  =  N 
o  o 


M  1/2T 

I  I  ’ 

p=l  -1/2T  2N  +  K  (f) 

r  0  T)W 


df. 


(4-129) 


The  optimum  time- invariant  filter  has  transfer  function  given  by 
Eq.  (4-59),  and  the  resultant  minimum  estimation- error  is  given  by  - 
Eq.  (4 -60b)  which  reduces  to' 


<e  >  =  N 
o  o 


M  1/2T 

I  / 


yf> 


df  . 


(4-130) 


p=l  -1/2T  2N  +  ^  K  (f) 
r  o  2  p 

From  these  last  two  formulas,  it  is  clear  that  for  low-noise  conditions, 

the  optimum  time-varying  filter  provides  a  3  dB  improvement  over  the 

optimum  time-invariant  filter,  regardless  of  the  specific  shapes  or 


differences  amongst  the  M  PSD's,  and  regardless  of  the  number  (M)  of 
signals  multiplexed;  i.e.,  there  is  an  independent  3  dB  improvement  for 
every  component  signal  x^. 

We  will  consider  the  FDM  signal  again  in  the  next  section  vil  re  we 
employ  the  HSR  to  gain  some  additional  insight  into  the  structure  of  the 
optimum  filter. 


iv)  Pulse-amplitude  modulated  signals.  The  PAM  signal  (Model  (2)  of 
Chapter  II), 

CO 

X(t)  =  -T— -  I  an6(t-nT)  (4-151) 

1  n=-» 

is  one  of  the  simplest,  and  yet  one  of  the  richest  examples  of  a  CS 
process.  Every  PAM  process  admits  a  first  order  TSR — in  fact,  as  shown 
in  Sec.  2  of  Chapter  III,  all  T-CS  processes  (in  H„)  can  be  represented 
in  terms  of  T-CS  PAM  processes  with  full  duty-cycle  rectangular  pulses, 
and  such  PAM  processes  have  constant  means  * nd  \rariances.  Furthermore, 
as  shown  in  Sec.  5  of  Chaprer  III*  all  T-CS  processes  can  also  be 
represented  in  terms  of  WSS  PAM  processes  with  bandlinitcd  (tol*l/2T,l/2Tj) 

i  u*  Kr  now. 
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pulses.  We  now,  use  these  two  types  of  PAM  processes  (those  where 
either  the  pulse  4  or  its  Fourier  transform  *  are  rectangular)  to 
demonstrate  three  interesting  results  concerning  optimum  filtering 
of  CS  processes. 

Before  proceeding,  however,  we  state  here  the  simplified  versions 
of  Eqs.  (4-58),  (4-60a)  for  estimation-errors  of  arbitrary  PAM  signals 
in  additive  white  noise: 


<e  > 

o 


f  j*(f)|2A(f) 

- - 7— - - df 

N  +  i  |*(f)|2A(f) 


*  -  f  |*(f)!2A(f) 

<e  >  =  N  /  — - ? - 

°  °-  N D  ♦  £  j*(f)|2A(f) 


df 


where 


A(f)  =  — — 
<P2(0) 


E{a  a  }e 
n+m  m 


-j2imTf 


|*(f)|2=  l  |$(f+n/T)|2.  (4-132) 

n 

(1)  Intuition  can  sometimes  be  misleading.  In  particular,  one  might 
intuitively  expect  that  a  cyclostationary  process  with  constant  mean 
and  variance  would  be  "almost"  stationary,  and  that  no  substantial 
improvement  would  result  from  using  periodic  filters  for  estimation 
of  such  processes.  As  a  counterexample,  consider  PAM  with  full  duty- 
cycle  unit-energy  rectangular  pulses.  This  process  admits  a  1st  order 
TSR,  and  has  constant  mean  and  variance.  If  the  PSD  K(f)  of  the  WSS 
process,  whose  samples  {a^}  are  the  pul st- amplitudes,  is  bandlimited 
tc  (- I/2T, 1/2TJ ,  then  from  Eq.  (4-132),  we  obtain  the  estimation-errors: 
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1/2T  N  K(f) 
<e  >  =  /  u 

n  * 


d£ 


•1/2T  N  +  K(f) 
o 


<e  >  =  } 

n  *- 


1/27  KlQK(f)  [sirnr  (Tf-n)/7r(Tf-n)]2 


n=-°°  -1/2T  Nq  +  K(f)[simr(Tf-n)/Tt(T£-n)]2 


df 


(4-135) 


Now,  for  low  noise,  <e  >  =  N  /T,  and  for  any  number  J  there  is  an  N 

0  0'  o 

small  enough  to  guarantee  that  <eQ>  >JNq/T.  For  example,  an  improvement 

factor  I  >  J  =  10  results  with  a  signal-to-noise  ratio  of  o2T/N  =  104 

o 

where  we  have  chosen  K(f)  =  a2T,  f  e  [-1/2T.1/2T] . 

(2)  Another  situation  where  intuition  might  lead  one  astray  is  in  the 
case  of  high  noise.  One  might  expect  that  signal-to-noise  ratios  as  low 
as  unity  would  degrade  time-varying  filter  performance  to  the  point  where 
improvement  over  that  of  the  optimum  time-invariant  filter  would  be 
negligible.  As  a  counterexample,  we  have  the  results  for  CSK.  In  addition, 
we  again  consider  PAM  but  this  time  with  unit-energy  ideal  lowpass  pulses 
with  bandwidth  M/T.  From  tq.  (4-132),  w'  "SI)  K(f)  constant  for 
f  e  [-1/2T,1/2T] ,  these  equations  reduce 


<e  >  = 

°  *2 


<j2o2  /M 
s  n 


s  n 


+  a2/M 
s  n 


-v  °-°2 

<e  >  = 

'•  O  9  o 

♦  az 
s  n 


(4-134) 


where  c j-  is  the  time-averaged  signal  variance,  and  a2  =  N  M/T  is  the 
s  no 

noise  variance.  Notice  that  the  effective  noise  variance  for  the  time- 
varying  filter  is  reduced  by  the  factor  1/M.  If  we  fix  the  signal-to¬ 
ne  ise  ratio  at  unity  (o2  =  a2),  then  the  improvement  factor  is  I  =  (M+l}/2. 
If  M  is  substantially  greater  than  unity  (M  =  1  means  transmission  at  the 
Nyquist  rate  and  renders  the  PAM  signal  WSS),  then  the  timc-v3rying  filter 
will  provide  substantial  improvement  in  performance  over  that  of  the 
t ime- intar i ant  filter  for  this  high  noise  situation. 
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(3)  In  contrast  to  the  two  above  results,  the  following  result  is  very 
much  in  agreement  with  intuition.  By  considering  PAM  with  less  than 
full  duty-cycle  rectangular  pulses,  we  find  that  improvement  increases 
as  the  signal  becomes  "more"  cyclostationary.  Specifically,  consider  a 
unit-energy  rectangular  pulse  of  width  T/M,  and  uncorrelated  pulse- 
amplitudes,  so  that  the  Fourier  transform  of  the  pulse  is  given  by 

♦  (£)  •  /t7m  , 

(irfT/M) 

and  the  one-dimensional  correlation  matrix  is  given  by 


A(f)  *  To2 


where  o2  is  the  time-averaged  signal  variance.  Now,  using  Eq.  («'  ’32) 
wc  obtain  the  estimation-errors: 


°s2VT 

<0  >  =  •— =-= - 

0  o2  ♦  N  /T 
S  0 

«  N  o2l4>(f)|2 

>  .  t  o  S 1  '  1  ,  „ 

<e  >  *»  J  - - df 

0  -«  N  +  o2 1 4* (f )  | 2 


»  N  o2|4>(f)  I  2 

>  /  df 

N  +  g2T/M 
0  s 

a2  N  /T 

S  0 


(4-135) 


°|/M  ♦  Nq/T 


(4-136) 


Hence,  for  low  noise,  we  have  an  improvement  factor  of  I  >  M.  Clearly, 
as  M  is  increased,  the  rectangular  pulses  become  narrower  so  that  the 
cyclostationarity  is  enhanced,  and  improvement  is  increased. 
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v)  PAM  signals  through  random  channels.  We  now  consider  the  transmission 
of  PAM  signals  through  random  channels  consisting  of  either  a  constant, 
but  random  attenuation  and  additive  white  noise,  or  of  a  constant,  but 
random,  delay  and  additive  white  noise.  We  assume  that  the  pulse- 
translates  {«{> (t-nT) }  are  orthonormal. 

(1)  Random  delay:  We  assume  that  the  channel  consists  of  a  constant, 
but  random,  delay  0  with  PD!1  PQ(‘)»  and  additive  white  noise  with  PSD  Nq, 
and  we  assume  that  the  phase  of  the  received  signal  has  been  determined  by 
a  "tracking  receiver"  (see  Subsection  7b  of  Chapter  II  and  [40j),  We 
consider  the  problem  of  estimating  the  transmitted  signal  x  with  its 
original  phase,  assuming  this  phase  information  to  be  of  some  importance. 
For  this  LMS  estimation  problem,  we  have  the  correlation  functions 


k  (t,s)  =  k  (t,s)  ♦  N  6 (t-s) 
yy  ’  xx  '  o  v  * 

-  y  A  4»(t-nT)<j>(s-mT)  +  N  6 (t-s) 

L  n-m  v  ov  * 

n,m 

00 

fixy(t,s)  =  /  kxx(t+o,s)p0(o)dc 

=  l  An-m  ♦Ct"nT)*(s-mT) 

n,m 

where 

00 

?(t)  =  /  <Ht+o)p  (o)do  .  (4-137) 

-CO 

By  employing  the  solution  method  of  Theorems  (4-5),  (4-6),  (4-7)  we  easily 
obtain  the  solution  for  the  impulse-response  function  for  the  optimum 
filter: 

h  (t,f)  =  l  II  ♦(t-nT)^(i-nT) 
n,in 


(4-138) 


where  the  z-transform  of  {H^}  is  given  by  Eq.  (4-61).  Thus,  the  only 

difference  between  this  optimum  filter  and  that  for  the  case  where  there 

is  no  random  delay  in  the  channel,  is  that  this  optimum  filter  uses  the 

average  pulse  <jT  to  regenerate  the  signal  rather  than  using  the  exact 

pulse  4>.  The  weighting  factors  used  as  amplitudes  for  the  regenerated 

signal  are  the  same  for  both  filters.  Hence,  linear  estimators  do  not 

appear  to  be  very  useful  for  obtaining  phase  information. 

(2)  Random  attenuation:  We  assume  that  the  channel  consists  of  a 

constant,  but  random,  attenuation  G  and  additive  white  noise  with  ESI) 

o 

Nq,  and  we  consider  the  problem  of  estimating  the  transmitted  signal  x. 
for  this  problem,  the  correlation  functions  are 

kyy(t.S)  =  Gj  kx)((t,S)  .  N0J(t-S) 

kxy(t,s)  =  r  kIX(t,s),  (4-139) 

so  that,  from  Theorem  (4-7),  we  have  the  following  solution  for  the 
optimum  impulse-response  function: 

h0(t,f)  «  l  ,,n.m*(t-«,n*(T-»Tj  (4-140) 

n,m 

where  the  z-transform  of  (II  )  is  given  by  the  formula 
11(f)  =  G“«\(f)/(Gj  A ( f )  *  No) 

=  (T  A(f)/(G^  2A(f)  *  N(f))  (4-141) 

where 

N(f)  -  « 6J  -  (T  2)A(f)  ♦  No 


(4-142) 
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From  this  second  expression,  we  can  interpret  our  solution  as  the 
solution  to  the  problem  where  the  channel  attenuation  is  deterministic 
and  equal  to  the  mean  attenuation  Gq  ,  and  where  the  additive  noise  is 
the  sum  of  the  original  white  noise  and  the  signal  weighted  by  the 
variance  of  the  channel  attenuation  (G£  -  Gq  ).  This  result  parallels 
that  of  Maurer  and  Franks  for  optimum  filtering  of  WSS  signals  through 
random  channels  [63] . 

Note  that  these  results  for  PAM  signals  through  .random  channels  arc 
valid  for  arbitrary  CS  processes  if  A(f)  is  interpreted  as  the  correlation 
matrix  for  the  TSR  of  the  C-S  signal. 

vi)  The  video  signal.  An  interesting  example  of  a  CS  process  that  results 
from  scanning  is  the  video  signal  (Model  (9)  of  Chapter  II)  whose 
Karhunen-I.ocve  TSR  is  discussed  at  length  in  Subsection  2d,2f  of  Chapter  III. 
If  we  ignoi*e  the  frame -to -frame  correlation  in  the  video  signal,  because 
of  its  relatively  low-frequency  nature,  then  the  TSR  correlation  matrix 
is  diagonal 


A  (f)  =  ii  \ 
pq  PR  P 


(l-o): 


where  are  the  eigenvalues, 
the  eigenfunctions  (Eq.  (3-27)): 

x  -i7- j> 

P  1+Y2’ 


- 


c°s(*foy), 
f  sin(nfoY  t), 


- - -  (4-143) 

+  4asin(nTf) 

and  where  the  basis  function  (4»  )■  are 


p  odd 

t  c  [-T/2.T/2] 

p  even 


where 


;  1/y  ,  p  odd 

tan(trf  y  T)  =  !  p 
op 

-Y  >  P  even 


(4-144) 


In  these  expressions,  a  is  the  line -to -line  correlation  factor,  and 
f  is  the  inverse-time-constant  of  the  exponent! <1  correlation  within 
a  line. 


Mow,  from  Theorem  (4-5),  the  optimum  time-varying  filter  for  the 

video  signal  in  white  noise  has  impulse-response  function  given  by 

Eq.  (4-S6),  (4-61b),  and  results  in  an  average  value  for  the  periodically- 

varying  minimum  estimation-error  given  by  Eq.  (4-62b): 

«  1/2T  N  A  (f) 

<o  -  l  J  — —  df 

0  p-1  -1/2T  N  ♦  A  (?) 

1  o  pp 


N  «  A 

‘  ~  ,il  t»  »  N 

*  l  p  ol-a  J  1  p  ol+aJ 


(4-145) 


As  a  specific  example,  we  consider  a  squai'e  picture  format  with 
500  lines  and  identical  picture  correlation  along  the  horizontal  and 
vertical : 


-2*f  T/500 
a  =  e  o 


(4-146) 


where,  for  typical  pictui'e  material,  0.7  <  a  <  0.99.  We  assume  an 
effective  single-sided  bandwidth  of  5O0/2T  thereby  obtaining  a  ratio 
of  the  time-avcraged-signal -variance  to  the  noise-variance  equal 


p  =  T/SOONy. 


(4-147) 
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Now,  since  we  are  assuming  that  the  receiver  bandlimits  the  signal 
to  the  effective  bandwidth  of  S00/2T,  then  only  the  first  500  terms  in 
the  Karhunen-Loeve  expansion  will  be  significant  [2],  so  that  the  infinite 
sum  in  Eq.  (4-145)  can  be  replaced  with  the  first  500  terms.  Using  this 
truncated  sum,  we  obtain  (from  digital -computer  evaluation)  the  plots 
of  minimum  estimation-error  vs.  signal-to-noise  ratio  for  various  values 
of  line-to-line  correlation  a  shown  in  Fig.  (4-17).  The  horizontal  lines 
in  this  figure  can  be  interpreted  as  the  error  which  results  from  an  ideal 
low-pass  filter  with  cutoff  frequency  500/2T.  Relative  to  this,  the 
optimum  time-varying  filter  providos  a  10  dB  improvement  at  low  signal- 
to-noise  ratio  (p=10  dB)  and  high  line-to-line  correlation  (a  =  .98). 

As  the  signal-to-noise  ratio  increases  and  the  correlation  decreases,  the 
improvement  eventually  becomes  negligible.  Furthermore,  for  all  practical 
values  of  signal-to-noise  ratio  and  line-to-line  correlation,  the  improvement 
over  the  optimum  time-invariant  filter  (not  shown  here)  is  negligible. 

This  absence  of  significant  improvement  can  be  predicted  in  advance  since 
the  constraint  of  Eq.  (4-146)  on  correlation  renders  the  CS  video  signal 
"almost  stationary".  This  low  "degree  of  cyclostationarity"  becomes 
obvious  with  the  aid  of  Figures  (1-3),  (1-6)  of  Chapter  I.  The 
autocorrelation  function  for  the  video  signal  is  the  product  of  the 
functions  shown  in  these  two  figures,  and  when  Eq.  (4-146)  is  satisfied 
with  the  order-of-magnitude-of-a  =  1,  then  the  cyclic  variations  (along 
the  lines  parallel  to  the  t  =  s  axis)  of  the  blocks  in  the  factor  shown 
in  Fig.  (1-5)  are  almost  totally  attenuated  by  the  narrow  fences  in  the 
factor  shown  in  Fig.  (1-6).  The  cyclic  character  of  this  CS  process  will 
only  be  significant  when  the  width  of  these  fences  becomes  comparable 
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to  (or  greater  than)  the  width  of  the  blocks;  i.e.,  when  f  T  and  a  are 
both  much  less  than  unity*  which  signifies  a  video  picture  composed  of 
nothing  more  than  slightly  correlated  horizontal  lines;  i.e.,  a  PAM-like 
signal  with  square  pulses  of  one  line-width.  These  results  have  been 
confirmed  by  digital-computer  evaluation  of  <«:  >  and  comparison  with 

'Vs 

<eQ>,  where  <eQ>  is  given  by  Eq.  (4-60u),  with  M  =  SOO. 
vii)  Time-division-multiplexed  signals.  W c  consider  here  the  TDM^ 
scheme  (Model  (3)  of  Chapter  II)  for  time-division-multiplexing  various 
(say  M)  signals  (x^)  together: 


oo  m 

x(t)  =  l  l  xn(t)w  (t-nT) 

n=-»  p=l  p  p 

where  (w  }  are  the  gate  functions 
P 


(4-148) 


w  (t)  - 
P 


1  V  t  e  ((p-l)T/M,pT/M)  -  Tp 
0  V-  t  t  T, 


(4-149) 


The  Karhunen-Loevc  TSR  for  this  TDMj  signal  is  discussed  at  length  in 
Subsections  2d,2f  of  Chapter  III.  If  we  assume  that  the  component 
processes  {x^}  are  uncorrelated  and  each  have  autocorrelation  functions 
which  satisfy  the  condition 


k  (t+rT-x)  =  aPk  (t-T) 

pv  t  r  p 


V-  t:,r  s  [0,T/M] 


(4 - 1 SO) 


then,  as  shown  in  Subsection  2f,  the  TSR  correlation  matrix  is  diagonal, 

and  if  ap  =6  then,  from  a  simple  extension  of  Theorem  (4-5)  and 
n-m  nm  1 

l:q.  (4-63),  the  optimum  filter  has  impulse- response  function 


M 


Ap 


Vt>T)  =  l  l  l  p  -q 

u=l  q=l  n=-“  N  ♦  A*  ^ 

*  1  A  /I 


$P(t-nT-pT/M)4>p (x-nT-pT/M) ,  (4- 

M 
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and  results  in  minimum  estimation-error 


.  M  « 

<V  -  T  l  l 


N  AP 
. ?-...cl 


T  L  U  d  > 

1  p=l  q=l  N  +  Xp 
v  M  o  q 


(4-152) 


where  {$p;  q  *  1,2,...}  and  (Ap;  p  «  1,2,...}  are  the  eigenfunctions  and 
M  4 

eigenvalues  for  the  kernel  kp(*,*)  on  the  interval  (0,T/MJ. 

Now,  if  we  choose  triangular  autocorrelation  functions  (kp)  with 
widths  {2Tp}  then,  as  shown  in  Subsection  2d,  the  eigenfunctions  and 
eigenvalues  are  given  by  the  formulas 


<i>P(t)  =  cos(wPt) 

4  4 

AP  =  2k  (0)/r  (wP)2  (4-153) 

q  p  '  p  q 


where 

2Mt 

tan (wpT/2M)  =  -  — p[(2M/u>PT)2  +  1]1/2  '  (4-154) 

q  up  T  T  q 

q 

provided  that  T/2M  <  <  T(l-1/M). 

Also,  using  a  simple  extension  of  Eq.  (4-60b)  results  in  the 
following  formula  for  the  minimum  estimation-error  resulting  from  the 
optimum  time-invariant  filter: 


'V 


■  / 


j>!  l  Kpcf)|2xP 


fLflSL 


1 


N  ♦  sp  J  l  |*P(f}|2XP 
0  F  p=l  q=l  q  q 


df. 


(4-155) 


Evaluation  of  the  performance  index  I  for  this  TDMj  signal  requires 
numerical  evaluation  of  Eqs.  (4-152) -(4-155),  and  has  not  been  carried 
out.  However,  judging  from  the  TDM-of-PAM  results  of  example  ii),  it 
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is  expected  that  significant  low-noise  improvements  will  result  if  the 
individual  variances  and/or  correlation-widths  of  the  component 
processes  {x^}  differ  significantly. 

c)  Examples  employing  the  harmonic  series  representation, 
i)  Frequency-division-multiplexed  signals .  It  was  shown  in  the  third 
example  in  the  previous  subsection  that  low-noise  improvements  of  3  dB 
can  be  obtained  if  optimum  time-varying  filters,  rather  than  optimum 
time-invariant  filters,  are  used  for  FDM  signals.  This  result  can  be 
rederived  using  the  HSR  instead  of  the  TSR  previously  used.  However, 
we  will  employ  the  HSR  here  to  study  only  the  structure  of  the  optimum 
time-varying  filter. 

As  shown  in  Section  3d  of  Chapter  III,  the  FDM  signal 
M 

x (t)  =  l  X  (t)cos(w  t  ♦  2npt/T)  (4-156) 

P=1  F 


w  =  N2n/T 
o 

admits  an  HSR  with  correlation  matrix 

Jyf).  IpI  =  hi  -  r  ♦  N 

0  ,  otherwise 


K  (f)  = 
pq 


(4-157) 


Thus,  from  Theorem  (4-9),  the  impulse-response  function  for  the  optimum 
time-varying  filter  for  the  FDM  signal  in  additive  white  noise  is  given 
by  Eq.  '1-65),  where  the  matrix  of  transfer  functions  H(f)  is  given  by 
liq.  (4-70),  which  reduces  to  the  formula 

ip  I  =  hi  =  r  ♦  N 


Kr(f) 


(f) 


Kr(f)  ♦  2N 


o 


(4-158) 


0 


i 


otherwise 


I 
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as  shown  in  Section  3e  of  Chapter  III.  This  optimum  filter  has  the 

;  1  '  .  .  i 

general  realization  shown  in  Fig.  (4-5).  But  due  to  the  fact  that  the 

,  f 

matrix  H  displays  the  symmetry: 

i  :  J 

/ 

Hpq  “  6)plhlHpp  ~  6ipllqlHp(-p)’ 

t^ie  structure  of  the  filter  can  be  reduced  to  that  shown  in  Fig.  (4-18), 

/ 

by  combining  paths.  This  structure  has  an  obvious  interpretation.  . 

<  '  •  i  i 

The  bank  of  input  modulators  demultiplex  the  FDM  signal  into  its 
component  signals,  which  tire  then  optimally  filtered  at  baseband5.  The 

l  .  J  * 

filtered  signals1  are:  then  remult ip 1  eked  together  by  the  output  bank  of 

i  •  1 

inodylators.  This  is  the' sam6  sequence  of  operations  performed  by  the 

I 

t  |  •  *  , 

optimum  time-varying  filter  for  the,TDM-PAM  signal,  and  emphasizes  the 
duality  referred  to  in  the  last  subsection.  ! 

,  I 

ii)  Amplitude  modulated  signals!  Amplitude  modulation  is  one  of  the 
most  frequently  used  signal  formats  for  communication.  In  fact,  the 

i  ,  1  ■  ’ 

FDM  signal  considered  in  the  previous  example  is  nothing  more  than  the 

i  * 

sum  of  M  AM  signals.  Hence,  the  Vesults  for  FDM  with  M  =  1  are  valid 

» 

fqr  AM  signals.  That  is,  for  the  AM  signal  (Model  (1)  of  Chapter  II) 

( 

I  ’  1  (  i  , 

,  x (t)  =  X  (t)cos(2nt/T) ,  ■  (4-159) 

O  i  1 

where  the  PSD  of  xQ  is, band limited  co  [-1/2T.1/2T] ,  the  optimum  time- 
varying  filter  has  the  structure  of  a  single  path  of  the  M-path  filter 

i 

for  FDM  shpwn  in  Fig.  (4-18),  and  provides  a  3  dB  improvement  in  perfoi'T 

1  i  J 

mance  over  that  of  the  optimum {time- invariant  filter.  Paralleling  the 
operations  performed  by  the  optimum  filters  for  FDM  and  TUM-PAM,  this 
optimum  t ime- varying  filter  demodulates  the  AM  Signal,  optimally 
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filters  it  at  baseband,  and  then  remodulates  the  filtered  signal. 

iii)  Sinusoid  with  random  amplitude  and  phase.  As  an  example  of  the 

/ 

general  results  obtained  in  Section  5c  for  IMS  smoothing  of  random 
periodic  signals  in  noise,  we  consider  here  the  random  sinusoid 


x(t)  =  asin(2trt/T)  ♦  bcos(2iTt/T) ,  (4-160) 

where  a,  b  are  zero-mean  random  variables  with  variances  a2,  a2  and 

d  D 

crosscovariance  o2^.  This  process  admits  a  first  oi'der  HSR  with 
correlation  matrix  elements 


*.1.1  ’  %  ■  -  °i'<  ♦  Hb/2 

K-ll  ‘  Kl-1  *  °a/4  '  °b/4 

K0±l  “  *±1G  3  Koo  =  °* 


(4-161) 


Now,  from  Theorem  (4-10)  the  optimum  time-varying  smoother  for 
this  signal  in  additive  white  noise  has  impulse-response  function  given 
by  Eq.  (4-74),  and  results  in  the  time-averaged  minimum  estimation-error 
given  by  Eq.  (4-76),  which  reduces  to 


<c  >  =  (2N  /.NT) 


A  +  B 


A  +  2B  ♦  4N  /NT 


where 


A  =  —•  (o2o2  -  o4  ) 
N  a  b  ab 


B  =  o2  +  a? 
a  b 


(4-162) 
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From  Theorem  (4-11),  the  optimum  time-invariant  smoother  has 
impulse  response  functipn  given  by  Eq.  (4-77),  and  results  in  the 
estimation  error  given  by  Eq.  (4-78),  which  reduces  to 

<e  >  =  (2N  /NT)  - - -  .  (4-3.65) 

B  +  4N  /NT 
o 

From  these  last  two  formulas,  it  is  clear  that  for  low-noise  conditions, 
the  performance  index  becomes 

A  +  2B 
1  ~  A  +  B 

and  is  a  maximum  of  3  dB  when  A  =  0.  But  A  =  0  iff  a  a  =  a2 

a  b  .ab 

iff  a  and  b  are  proportional  iff  the  phase  of  the  random  sinusoid  is 
deterministic.  This  result  is  exactly  what  one  would  expect:  The  time- 
varying  filter  outperforms  the  time- invariant  filter  by  the  greatest 
margin  when  the  sinusoid  displays  its  highest  degree  of  cyclostationarity; 
i.e,,  when  its  phase  is  completely  known. 

d)  Examples  gmploying  the  Fourier  series  representation, 
i)  Amplitude-modulated  signals .  In  the  previous  two  subsections  we 
employed  the  TSK  and  HSU  to  derive  the  structure  of  the  optimum  time- 
varying  filter  for  AM  signals,  and  to  show  that  the  performance  index  is 
a  maximum  of  5  dB  for  low-noise  conditions.  These  results  can  also  be 
derived  using  the  FSR  as  briefly  outlined  here;  The  autocorrelation 
function  for  the  AM  signal 

x(t)  =  xQ(t)cos(2»t/T) 

where  xQ  is  band  limited  to  (-1/4T, 1/4TJ  admits  a  second  order  FSU  with 


coefficient  functions 
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Vf)  »  i  Ko(frl/T)  ♦  \  K0(f+1/T) 

Cj(f)  «  C_j(f)  =  0 

c2(f)  c  C_2(f)  =  i  KQ(f)  (4-164) 


where  K  is  the  PSD  for  x  . 
o  o 

Now,  from  Theorem  (4-13),  the  optimum  time-varying  filter  admits 
the  second  order  FSR  of  Eqs.  (4-83)-(4-8S) .  Substituting  Eq.  (4-164) 
into  these  formulas  yields  formulas  for  the  transfer  functions  for  the 
time-invariant  filters  in  the  optimum  txme-varyirg  filter  structure  of 
l;ig.  (3-14a).  With  these  formulas,  this  structure  can  be 

reduced  to  the  simple  demodulator- filter-modulator  structure  identified 
in  oxamplo  ii)  of  the  last  subsection. 

Also,  this  optimum  time-varying  filter  results  in  the  minimum 
estimation-error  given  by  Eq.  (4-88)  of  Theorem  (4-13).  Substitution 
of  Eq.  (^ - 164)  into  this  fonrula  yields  the  following  expected  formula: 


<c  > 
0 


1/2T  N  K  (0 
j  o  ov  * 

-1/2T  2N  -*  K  (f) 
o  o 


df  . 


(4-165) 


Note  that  the  above  result  is  valid  as  long  as 
to  [-1/2T, 1/2T] ,  but  the  solution  method  of  Theorem 
for  xQ  bandiimited  to  (-1/4T, 1/4TJ . 
ii)  Bandpass  PAM.  Consider  the  following  bandpass 


x  is  bandiimited 
o 

(4-13)  is  only  valid 


PAM  signal 


x(t)  «*  >/2T  l  a  $(t-nT), 
n 


where 
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$(t)  *  cos (u  t)  ZBiUll* II  ,  (4-166) 

°  (nt/4T) 

where  u>o  is  an  integer-multiple  of  2ir/ T.  Now,  if  we  consider  the  {an> 
to  be  the  sample  values  (xo(nT))  of  a  process  which  is  bandlimited  to 
[-1/4T,1/4T],  then  bq.  (4-166)  can  be  reexpresscd,  using  the  sampling 
theorem  (Theorem  (2-2)),  as 

x(t)  =  2xo(t)cos(«Qt) 

and  is  therefore  equivalent  to  the  AM  signal  of  the  previous  example. 

Hence,  the  optimum  filter  and  estimation-error  derived  there  arc  valid 
here.  However,  the  most  meaningful  filter  structure  results  from  using 
the  TSR  (this  structure  consists  of  one  path  of  the  M-path  structure  of 
Figure  (4-16))  rather  than  the  FSR  (or  the  HSR).  In  fact,  the  solution 
obtained  using  the  TSR  is  valid  even  if  the  an  are  not  the  sample  values 
of  n  process  which  is  bandlimited. 

iii)  Improvement  for  general  PAM.  In  subsection  6b,  we  presented  the 
general  formulas  of  bq.  (4-132)  for  estimation-errors  for  arbitrary 
PAM  signals  in  additive  white  noise.  We  will  now  reexpross  these  formulas 
in  terms  of  FSR  coefficient  functions  in  order  to  gain  insight  into  the 
relationship  between  improved  performance  and  the  FSR  coefficient  functions. 

Substituting  the  TSR  of  bq.  (5-2)  for  the  autocorrelation  function 
for  the  PAM  process  into  the  formula  of  bq.  (3-101)  for  the  p^'  FSR 
coefficient  function  yields  the  formula 


c  (t)  =  ~  l  A  0(t-rT)e’j2,,»,t/T 
*  r 
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where 

oo 

o(t)  -  /  *(t+T)+(t)dt. 


Thus,  the  Fourier  transform  C  of  c  takes  the  form 

P  P 


Cp(f)  =  £  A(f)*(f+p/2T)**(f-p/2T).  (4-167) 


Now,  from  this  formula,  we  obtain  the  relation 


£  A(f)|4>(f+P/T)|2  =  CQ (f+p/T)  =  |Cp(f+p/T)|2/C0(f)  .  (4-168) 


Substituting  this  into  the  formulas  of  Eq.  (4-132)  yields  the  new  formulas 


*  1  I 


1/2T  NoC0(f.p/T) 


df 


<V 


P  -1/2T  No  *  Jp(f)C0(f+p/T) 
1/2T  N  Cn (f+p/T) 

l  I  - df 

p  -1/2T  No  +  CQ (f+p/T) 

|C  (f+q/T) | 2 

j  (f)  5  £  __S -  . 

1  q  C0(f)CQ (f+p/T) 


(4-169) 


Now,  from  these  formulas,  it  is  clear  that  for  every  value  of  the 
index  p  for  which  J  (f)  >  1,  \l  f  e  [-1/2T, 1/2T] ,  there  is  a  contribution 
to  the  overall  improvement-in-performancc  (of  the  optimum  time-varying 
filter  over  the  optimum  time-invariant  filter).  Furthermore,  J  (f)  is 
a  measure  of  the  relative  magnitudes  of  the  zeroth  and  pth  order 
FSR  coefficient  functions.  Thus,  we  have  here 

a  positive  relationship  between  large  higher  order  FSR  coefficient  functions, 
and  large  improvements. 


e)  Examples  employing  the  i nvcrsc-opcra tor- feedback-system  method. 
In  subsec.  5o  we  presented  four  (.lasses  of  optimum  filtering  problems  to 
which  lire  inverse-operator-fecdbacksystem  (10FS)  method  of  solution 
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applies.  We  now  illustrate  these  with  solutions  for  specific  problems, 
i)  Periodically  attenuated  WSS  signal  in  WSS  noise.  Consider  the 
problem  of  optimally  filtering  a  bandpass  WSS  signal  with  PSD 


K  (f) 
xx v  J 


o2t/2,  | |f |  -  fj  <  1/2T 


(4-170) 


0  ,  otherwise 

which  has  been  transmitted  through  a  channel  with  slowly-varying  periodic 

attenuation,  1  +  acos(2*t/T) ,  and  additive  WSS  white  noise  with  PSD  N  . 

o 

This  problem  belongs  to  class  (1)  of  Subsec.  5e  with  M  =  2TfQ  »  1 
(assumed  to  be  an  integer),  and  N  -  1  in  Hq.  (4-97).  Thus,  the  impulse- 
response  function  for  the  optimum  time-varying  filter  is,  from  Oqs.  (4-98), 
(4-99),  (4-101), 

1  +  acos(27iT/T) 


h  (t,x)  - - k  (t-x) 

0  N  +  a2T(l+a2/2)/2  xx 


(4-171) 


O  X 

and  has  the  realization  shown  in  Pig.  (4*19).  Furthermore,  the  time-averaged 
value  of  the  minimum  estimation-error  is,  from  Hq.  (4-29) 


<c  >  =  a2  - 


1 


T/2 


No  *  o^T(l+a2/2)/2  -T/2-®' 


/  /  (l+acos(2irs/T))?k2  (t-s)dsdt 

*  -r/->  „  xx 


1  +  p  (l+a2/2) 


(4-172) 


where  p  is  the  signal-to -noise  ratio  (p  =  o~T/2N  ), 


The  estimation- error  which  results  from  the  optimum  time-invariant 
filter  (i.c.,  that  for  the  problem  where  the  periodic  attenuation  is 
replaced  with  its  average  value)  is,  from  Hq.  (4-27) 


<o  >  = 
o 


1  +  p 


(4-173) 
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Hence,  there  is  a  low-noise  improvement  factor  of  I  =  (l+a2/2), 
and  I  <  3/2,  assuming  |aj  <  1. 

It  is  interesting  to  note  that  the  optimum  filter,  Fig.  (4-19) 
decomposes  into  the  cascade  connection  of  a  periodic  multiplier,  which 
attempts  to  equalize  the  periodic  attenuation,  and  a  Weiner  filter 
which  optimally  filters  the  equalized  signal.  It  should  be  emphasized, 
however,  that  the  multiplier  is  not  an  equalizer  in  the  true  sense,  and 
the  use  of  an  exact  equalizer  would  result  in  inferior  performance, 
ii)  AM-type  CS  signal  in  WSS  noise.  For  the  class  of  optimum-filtering 
problems  where  a  CS  signal,  which  is  the  product  of  a  deterministic  T- 
periodic  signal  q  and  a  WSS  process  z  with  PSD 

o2T  ,  |fj  <  1/2T 

K  (f)  =  2  "  (4-174) 

0  ,  otherwise, 

is  transmitted  through  a  channel  with  additive  WSS  white  noise  with 
PSD  Nq  (class  (2)  of  Subsection  5e),  the  optimum  time-varying  filter  is 
easily  shown  (using  Hqs.  (4-98),  (4-99),  (4-103))  to  have  impulse-response 
function 

h0(t,T)  =  q(t)q(T)kxx(f  t)/(Qo2T  ♦  Nq).  (4-175) 

Thus,  the  minimum  estimation-error  is,  from  Fq.  (4-32), 

Q°*N/t 

<e  >  =  - — -  , 

0  Qo2  ♦  N  /T 
z  o 

where 

Q  -  l  h„l2  .  (-1-176) 

n 

where  {q}j}  are  the  Fourier  coefficients  for  the  T-periodic  function  q. 
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Also,  from  Eq.  (4-27),  theoptimum  time-invariant  filter  results 

in  the  estimation-error 

'  o2h  | 2N  /T 

0  n  o2|q  I2  +  N  /T 

z,nn'  o 

(where  we'  have  used 

^cx(f)  =  ^  Kxx(f+n/T),)  (4-177) 

n 

Now,  for  low-noise  conditions,  we  have 

<e  >  =  N  /T  • 
o  o 

<0  >  >  N  2N/T  (4-178) 

o  o 

where  No  <<  o2|qn|2T  for  ]n|  <  N,  so.  that  there  is  a  lew-noise  improvement 
factor  of  I  >  2N.  Hence,  there  can  be  significant  improvement  if  the 
periodic  factor  q  has  high  harmonic  content  as,  for  example,  in  a 
rectangular  gating  function. 

One  should  be  careful  in  interpreting  this  result.  For  example,  if 
q  is  the  rectaj.gular  gating  function  shown  in  Fig.  (4-20a),  then  one 
might  reason  that  the  optimum  time-varying  filter  obviously  performs  much 
better  than  the  optimum  time-invariant  filter  since  the  former  passes  zero 
noice  to  the  output  during  the  majority  of  every  T-second  interval  (when 
the  signal  x  is  itself  :sro),  whereas  the  latter  can  not  behave  in  such  a 
time-varying  fashion.  That  this  reasoning  is  invalid  is  clear  when  we 
consider  the  new  gating  function  shown  in  Fig.  (4-20b).  The  improvement 
factor  is  the  same  for  this  gating  function  as  it  is  for  that  in  Fig.  (4-20a), 
yet  the  above  argument  obviously  does  not  apply,  since  the  optimum  time- 


■•■*v^*'*'>  *  •fgr-'t'TZ  <7-* •*.*.->  j  •<•»  *>w,'-^."  *» 
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varying  filter  now  passes  zero  noise  to  the  output  only  during  a  very 
small  portion  of  every  T-second  interval.  Hence,  it  must  be  concluded 
that  the  improvement  is  simply  due  to  the  fact  that  the  time-invariant 
filter  can  not  reproduce  the  sharp  features  in  the  gated  signal  without 
passing  high-frequency  noise,  whereas  the  time-varying  filter  easily 
reproduces  the  sharp  features  by  employing  a  gate  of  its  own. 
iii)  WSS  signal  in  white  CS  noise.  Consider  the  class  of  optimum- 
filtering  problems  where  a  WSS  signal  with  PSD 

ojr,  |f|  <  1/2T 

0,  otherwise 

is  transmitted  through  a  channel  with  additive  white  CS  noise  with 
autocorrelation  function 


Kxx^ 


(4-179) 


knn(t,s)  = 


(4-180) 


These  problems  belong  to  class  (3)  of  Subsection  5e.  Thus,  the  impulse- 
response  function  for  the  optimum  time-varying  filter  is  given  by 
Eqs.  (4-98),  (4-99),  (4-105): 


h  (t,x)  =  q"2(x)  - — —  k  (t-T), 

0  1  <  !>o2T  xx 


(4-181) 


and  the  resultant  estimation-error  is.  from  Eq.  (4-29), 

,  T/2  » 

<0  >  =  o2  -  r:  /  /  q"2(s)k2  (t-s)dsdt 

0  x  1  -T/2-"  X 


1  +  Po2T 


(4-182) 


29S 


where 

P  -  Y  /T/V2(t)dt. 

1  -T/2 

The  estimation-error  which  results  from  the  optimum  time-invariant 
filter  is,  from  Eq.  (4-2S), 


"V 


o2 

X 

1  +  02T/Q 


(4t183) 


where 

T/2 

Q  -  l  I  q2(t)dt. 

1  -T/2 

Hence,  the  low-noise  improvement  factor  is  I  =  QP.  For  example, 

if  the  noise  intensity  q2(t)  switches  back  and  forth  every  T/2  seconds 

from  N  to  aN  (a  <  1),  then 
o  o  = 


QP  «*  i  (1  ♦  a)(l  ♦  1/a), 


(4-184) 


and  the  improvement  exceeds  3  dB  for  a  <  1/5. 

iv)  Doppler-shifted  Poisson  pulse-train.  We  consider  here  the  problem 
of  optimally  filtering  a  Doppler-shifted  Poisson  pulse-train  in  additive 
white  noise.  We  model  the  signal  x  as  an  asynchronous  PAM  signal  which 
has  been  subjected  to  a  periodic  time-scale  transformation  t'  =  t  +  q(t) 
(Model  (8)  of  Chapter  II): 

CO 

X (t)  =  l  a  j(t*q(t)-t  ) ,  (4-185) 

n=-“ 

where  the  original  (before  Doppler)  pulse-occurrence-times  {t  }  form  an 
order  sequence  distributed  according  to  the  Poisson  counting  process  with 
constant  rate  parameter  and  where  (a  }  is  a  zero-mean  WSS  sequence. 
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The  PSD  for  the  original  process  without  Doppler  (q(t)  =  0)  is 

Kzz(f)  =  Vll$(f)l2'  (4'186) 

so  that  if  the  pulse  $  is  a  sine  pulse  with  Fourier  transform 
;t,  I f |  <  1/2T 

*(f)  =  ;  "  (4-187) 

;0,  otherwise, 

then  this  optimum-filtering  problem  belongs  to  class  (4)  of  Subsec.  Se 
where  g(t)  »  t  +  q(t).  Thus,  the  impulse-response  function  fpr  the 
optimum  time-varying  filter  is,  from  Hq.  (4-98),  (4-99),  (4-107): 

h  (t,T)  - - — ^ - k  (g(t)-g(x)),  (4-188) 

0  Pp  <j2T2  +  N  22 

o  a  o 

where 

a  i  1/2T 

P  -  f  I  1/g' (t)dt 
1  -1/2T 

and  has  the  realization  shown  in  Fig.  (4-21).  The  structure  of  this 
optimum  filter  has  the  interesting  configuration  consisting  of  an  inverse 
time-scale  transformation,  followed  by  a  time-invariant  Weiner  filter 
and  a  time-scale  transformation  at  the  output.  Thus,  the  filter  removes 
the  periodic  fluctuations  from  the  CS  signal,  optimally  filters  the 
resultant  WSS  signal,  and  then  re- inserts  the  periodic  fluctuations.  This 
sequence  of  signal  processing  operations  parallels  the  demodulator-filter- 
modulator,  and  demultiplexor-filtcr-multiplexor  sequences  derived  in  many 
of  the  previous  examples. 

The  time  averaged  value  of  the  minimum  estimation-error  which  results 
from  the  optimum  time-varying  filter  is,  from  Uq.  (4-32), 


L 

Li 

0 

0 
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<e  >  = 

o 


N 

o 


Pu  a2T2  +  N 
o  a 


o 


i  /  k  (g(t)-g(t))dt 
1  -1/2T  zz 


U  o2T2N  /P 
o  a  o 

y  a2!2  +  N  /P 
o  a  o 


(4-189) 
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v)  Time-division-multiplexed  signals .  In  Chapter  II  we  introduced  the 
information  lossless  TDM.,  scheme  (Model  (5))  as  an  alternative  to  the 
information  lossy  TUM^  scheme  (Model  (3))  for  time-division-multiplexing 
various  signals  together.  The  TDM2  signal  is  formed  by  periodically 
time-compressing  the  component  processes  into  non-overlapping  time-slots, 
so  that  when  the  components  are  added  they  will  not  interfere  with  each 
other  (See  Fig.  (2-7)  of  Chapter  II).  If  the  M  component  processes  are 
statistically  independent,  then  it  can  be  shown  that  the  optimum  time- 
varying  filter  for  the  TDM2  signal  in  additive  white  noise  is  a  parallel 
connection  of  M  independent  filters,  the  p  of  which  is  the  optimum  time- 
varying  filter  for  the  p^1  time-compressed  component  process.  These  M 
filters  include  periodic  gates  at  their  inputs  so  that  the  p1"*1  filter 
admits  only  the  p^’  component  signal  (in  white  noise). 

Now,  the  impulse-response  friction  for  the  p^1  optimum  filter  must 
satisfy  the  orthogonality  condition 

CO 

/  k  (T,o)h  (t,o)do  +  N  h  (t.x)  =  k  v  (t,t) 

1  x  x  n  on  xx 

-®  p  p  1  1  p  p 

where 

kx  x  (t,s)  =  k2  2  (t+q(t+ (M-p)T/M)  -  s  -  q(s+(M-p)T/M) • 

'  P  P  P  P 

w(t-(p-l)T/M)w(s-(p-l)T/M) 


yy-^rm 
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w(t)  - 


where  z^  is  the  original  component  process  from  which  the  time-compressed 
version  x^  is  obtained,  and  where  w  is  the  periodic  gate  function 

1,  0  <  Jt-nTj  <  T/M,  V-  n 

0,  otherwise  , 

and  where  q  is  the  periodic  time-scale  compression  function  shown 
(as  p)  in  Fig.  (2-6)  of  Chapter  II. 

This  optimum-filtering  problem  is  a  combination  of  the  types  of 
problems  in  classes  (2)  and  (4)  of  Subsection  Se,  and  can  therefore 
be  solved  in  some  cases  using  the  IOFS  method  of  that  subsection. 


f)  Summary  and  Conclusions.  In  concluding  this  chapter  on 
optimum  filtering  of  cyclostationary  processes,  we  emphasize  the  intuitively 
pleasing  result  that  optimum  time-varying  filters  for  cyclostationary 
processes  can  always  be  decomposed  into  a  sequence  of  three  signal 
processing  opci'ations;  1.  The  cyclostationary  process  is  decomposed  into 
one  or  more  (jointly)  stationary  processes.  2.  The  stationary  component- 
processes  are  passed  through  time-invariant  filters  (often  Wiener  filters). 
3.  The  filtered  components  are  recombined  to  form  the  optimally  filtered 
cyclostationary  process,  in  most  of  the  examples  presented,  this  sequence 
of  signal  processing  operations  takes  one  of  the  following  specific 
forms:  demodulator-filter-inodulator,  dcinultiplexor-fi lter-multiplexor, 
inverse  time-scale  transformation--filter-‘ time-scale  transformation. 

For  those  particular  cyclostationary  processes  which  decompose  into  a 
single  stationary  process,  the  three  operations  are  more  accurately 
described  as:  1.  Removal  of  periodic  fluctuations.  2.  Time- invariant 
filtering.  3.  Re-insortion  of  periodic  fluctuations. 


I 


I 
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i  ,  i 

In  fact,  the, optimum  titae-varying  filters  derived  for,  FSX,  PAM, 

*  i  *  * 

TDM- PAM,  AM,  and  FDM  signals  in  additive  white  noise  all  consist  of 

1  i  1 

demultiplexor-demodulaton--time-ipvariant  Wiener  filter--raodulator- 

t  i  ' 

'■  multiplexor  structures.  ,  ,  ' 

Now,  if  the  optimum  filter  is  to  be  employed  in  a  receiver,  then  it 

i 

will  be  followed  by  a  demultiplexor-demodulator.  But  the  same  "baseband" 

!  '  •  1  (  I 

signal  can  be  obtained  more  directly  by  simply  eliminating  the  third 

1  .  '  !'  I  i 

*  operation  of  modulation  and  multiplexing  from  the  optimum  time-varying  ; 

i  i  ! 

filter,  thereby  significantly  reducing  the  complexity  of  the  overall  receiver. 

,  «  '  1  '  ! 

Hence,  the  optimum  linear  receiver  is  a  demultiplexpr-demodulator 

'  i  i 

followed  by  a  time-invariant  filter(s).  It  is  interesting  to  note  that 

,  .  i  1 

is  , 

this  optimum  linear  receiver  employs  the  same  two  types  qf  operations  as 

>  i 

*  J  / 

the  suboptimum  receiver  which  ignores  the  cyclic  character  of  the  • 

.  1  '  .  .  1 

i  cyclostationaVy  signals,  but  in  reverse 'order;  i.e.,  the  suboptimum  receiver 

i 

consists  of  an  optimum  time-invariant  filter  followed  by  a  ! 

i  ’  1  1 

demultiplexor-demodulator.  ,  .  <  ' 

* 

i  We  state  this  very;  practical  result  Here  as  a  theorem  to  be  referred 

'  1  ‘I  1  ‘  •  I 

to  as  tho  Optimum  Linear  Receiver  Theorem:  i 

THEORBM(4-14) :  Consider  the  class  of  signals  which  are  obtained  by 

•  i 

linear  modulation  and  multiplexing  of  W,SS  waveforms  and  data  sequences. 

If  the  additive  noise  to  which  these  signals  are  subjected  during  transmission 

:  i  5 

is  equivalent29to  WSS  white  noise  after  demultiplexing  and  demodulating, 

t  ■  1 

•  i  1 

then  the  optimum  (LMS)  linear  receiver  consists  of  the  demultiplexor- 
demodulator  followed  by  time- invariant  (continuous-  or  discrete-time) 

Wiener  filters.  '  : 

I  * 

29  In  this  theorem,  two  noises  are  "equivalent"  if  they  resul.  in  the  same 
orthogonality  condition  for  LMS  estimation. 
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The  proof  of  this  theorem  is  straightforward  provided  that  the 
linear  modulation-multiplexing  transformation  is  invertible— a 
requirement  which  must  be  satisfied  in  practice. 

The  essence  of  this  theorem  is,  as  discussed  above,  the  fact  that 
the  time- invariant  filtering  must  follow- -not  precede— the  demultiplexing 
and  demodulation. 

Another  result  worth  emphasizing  is  that  cyclostationary  processes 
whose  mean  and  variance  are  constant  can  still  possess  a  high  degree  of 
cyclostationarity  in  the  sense  that  optimum  time-varying  filtering  can 
yield  substantially  lower  mean* squared  error  than  optimum  time-invariant 
filtering. 

A  third  interesting  result  is  that  optimum  time-varying  filters 
for  eye lost at. ionary  processes  appear  to  yield  the  greatest  reduction  in 
inean-squared  error  (below  that  resulting  from  optimum  time-invariant 
filters)  when  the  signai-to-noise  ratio  is  high,  but  can  yield 
substantial  reductions  even  at  signal-to-noise  ratios  ;s  low  as  unity. 


Figure (4>1)  Communication-system  diagram 
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Figure(4-2)  Orthogonal  projection  in  two-dimensional  iluclidean 
space. 
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Theorems (4-6, 7),  tlie  transfer  functions  for  the  input  filters 


r  Theorems (4-6,7) ,  the  input  modulating  functions  are  compose; 


INVARIANT 

FILTERS 


sin(2nMt/T) 


l:i}*wre(-l-7)  Structure  of  optimum  filter  derived  usinj»  an  ! 
order  HSR.  (See  Theorcm(4-13) . ) 
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Figure(4-8)  System  diagram  of  optimum  filter.  (K^,  Kxy  are 
T-pcriodic  linear  systems.) 
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Figure (4-9)  Feedback  configuration  for  optimum  filter.  (K^  is  a 
time-invariant  linear  system,  p{»)  is  a  T-pcriodic  function,  and 
IKj,K9  arc  T-periodic  linear  systems.) 


averaged  minimum  cstimat 


q(t) 
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Figure(4-19)  Optimum  time-varying  filter  for  periodically 


attenuated  signal  in  additive  white  noise;  q(t)  *  l*acos(2nt/T), 

2 


il(f)  -  Kxx(f)/(No*(U|-)Kxx(f)). 
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inverse  Wiener  time-scaling 

time-scaling  filter 


Figure(4-21)  Structure  of  optimum  time-varying  filter  for 
periodically  time-scaled  processes;  H(f)  «  K2t(f)/(PQKsl(f’|tN0) . 


CHAPTER  V 


SUMMARY 


In  this  final  chapter  we  give  a  detailed  outline  and  summary 
of  the  contents  of  Chapters  II,  III,  IV  which  comprise  the  body  of 
this  dissertation.  It  is  hoped  that  this  combination  index,  outline, 
and  summary  will  help  to  offset  the  length  of  Chapters  II,  III,  IV. 
Following  this  summary,  wc  suggest  several  topics  for  further  research. 


1.  Summary  of  Chapters  II,  III,  IV. 


a)  Chapter  II:  Transformation,  Generation,  and  Modeling  of 
Cyclostationary  Processes.  The  second  chapter  of  this  dissertation 
provides  a  foundation  upon  which  many  of  the  developments  in  this 
dissertation  are  based,  and  upon  which  subsequent  studies  dealing  with 
cyclostationary  processes  can  be  based.  Specifically,  Chapter  II  is 
devoted  to  the  topics  of  generation  of  cyclostationary  processes,  and 
transformations  on  them.  Characterizations  of  the  various  classes  of 
transformations  outlined  below  arc  developed  and  illustrated  with 
numerous  models  for  cyclostationary  processes.  These  models,  also 
listed  below,  are  used  throughout  this  dissertation  for  illustrating 
various  theoretical  results.  Most  of  these  models  are  of  cyclostationary 
signals  commonly  used  in  communication  and  control  systems. 

Section  1  of  Chapter  II  is  introductory.  In  Section  2  we  consider 
transformations  which  are  linear.  After  ueriving  general  input-output 
relations,  we  individually  discuss,  in  some  detail,  the  three  subclasses 
of  linear  transformations  referred  to  as  time-invariant  filters,  periodically 
time-varying  systems,  and  time-scale  transformations.  The  impulse-response 
function  and  Zadeh's  system  function  are  introduced  as  means  for 
characterizing  linear  systems,  and  are  used  throughout  this  section. 
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In  Section  3,  we  briefly  discuss  multi-dimensional  linear 
transformations  as  a  means  for  characterizing  scanning  operations. 

In  the  fourth  section,  we  introduce  a  generalization  of  Wiener's 
Volterra  series  representation  for  characterizing  input-output  relations 
for  periodic  nonlinear  systems. 

Section  S  on  random  linear  transformations  parallels  Section  2  on 
deterministic  linear  transformations.  After  deriving  general  input-output 
relations  in  terms  of  a  generalization  of  Zadeh's  system  correlation 
function,  we  individually  discuss,  in  some  detail,  the  three  subclasses 
of  random  linear  transformations  referred  to  as  wide-sensc  stationary 
systems,  eye 1 os tat  ionary  systems,  and  random  time-scale  transformations. 

Section  6  on  random  multidimensional  linear  transformations  parallels 
Section  3,  The  major  application  of  the  theory  in  this  section  is  to 
random  scanning. 

The  final  section  on  random  nonlinear  transformations  is  an 
extension  of  Section  4  on  deterministic  nonlinear  transformations.  The 
major  issue  in  this  section  is  jitter.  This  random  disturbance  is 
characterized  in  general  terms,  and  shown  to  preserve  cyclostationarity 
under  fairly  liberal  conditions. 

The  specific  contents  of  this  chapter  arc  outlined  below: 

Section  1 :  Introduction 

Section  2:  Linear  Transformations. 

a)  Time-invariant  i'i  Iters. 

Theorem  f 2- 1 ) :  I'rcserv.u  ion  of'  cyclostationarity  by  time-invariant 
I'i  I  ter*.. 


I 


:  327’ 

’  !  i 

Theorem  (2-2):  Sampling  theorem  for  nonatationary  processes. 

•  i 

Theorem  (2-3):  Redaction  of  cyclostationary  processes  to 

* 

stationary  processes  Via  bandliraiting  filters. 

•  i 

b)  Periodically  time-varying  systems. 

Theorem  (2-4):  Generation  of  cyclostationary  processes  from 
stationary  processes  via  periodic  systems. 

•  i 

i 

Model  (1):  Amplitude-modulated  signals. 

Model,  (2):  Pulse-amplitude-modulated  signals. 

'  1  I 

Model  (3):,  Time-division-multiplexed  signals. 

Model  (4):  frequency-division-multiplexed ;signals. 

i 

c )  Time -scale  transformations . 

(  l  1 

Theorem  (2-5):  Generation :of  cyclostationary  processes  from 
stationary  processes  via  periodic  time-scale 
i  transformations.  , 

i  ■  ■ 

Model  (S):  Time-divisiou-multiplexed  signals. 

i  { 

Model  (6):  Doppler-shifted  processes.  ( 

l 

Model  ( 7(  1 :  Random  facsimile  signal. 

i  '  * 

Model  (8):  Asynchronous  pul  sc-ampl  i  tude- modulated  signals., 

i  * 

1  < 

Section  3:  Mill ti‘-d miens ion.il  linear  transformations  (scanning) 

I 

theorem  (2-0):  Genet -a ion  of  one-dimensional  cyclostationary 
processes  from  multi-dimensional  stationary 
processes  via  multi -dimensional  periodic  systems 

,  •  l 

Model  (D):  I  he  \ ideo  signal  (line  scanning). 

Model  M0|:  I  he  passive  radar  signal  (circular  scanning). 


I 
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Section  4:  Nonlinear  Transformations. 

Theorem  (2-7) :  Generation  of  cyclostationary  processes  from 
stationary  processes  via  periodic  nonlinear 
Vo 1 terra  systems. 

Model  (11):  Parametrically  amplified  signals. 

a)  Zero-memory  nonlinearities. 

Theorem  (2-8) :  Generation  of  cyclostationary  processes  from 
stationary  processes  via  periodic  zero-memory 
nonl incar i ties. 

b)  Nonlinear  modulation  of  synchronous  pulse-trains. 

Model  (12):  Prequcncy-shi ft-keyed  signals  (PSK). 

Model  (13):  Phasc-shi ft-keyed  signals  (PSK). 

Model  (14):  Pulse-width-modulated  signals  (PWM). 

Model  (15):  Pulse-position-modulated  signals  (PPM). 

Model  (16):  Digital  pulse-frequency-modulatcd  signals  (DPPM) . 

Model  (17):  Synchronous  M-ary  signals. 

Theorem  (2-9):  Cyclostationari ty  of  synchronous  pulse-trains 

nonlinearly  modulated  with  stationary  sequences. 

Section  5:  Random  linear  Transformations, 
a)  hie -sense  stationary  systems. 

Theorem  (2-10):  Preservation  of  widc-sense  stationarity  by 
random  t rans format i ons . 

Model  (IS):  Random  periodic  signals. 

Theorem  (2-11):  Conversion  of  deterministic  periodic  systems  into 

wide-sense-stat ionary  systems  via  phase-randomisation. 
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Theorem  (2-12):  Preservation  of  cyclostationarity  by  random 
transformations . 


b)  Cyclostationary  systems. 

Theorems  (2-13):  Generation  of  cyclostationary  processes  from 
(2-14) 

stationary  processes  via  random  transformations. 
Model  (19):  Random  periodic  attenuation  of  stationary  processes. 
Model  (20) :  Multiplication  of  a  stationary  process  by  a 


cyclostationary  process. 


c)  Random  time-scale  transformations. 

i)  Phase-randomization. 

Theorem  (2-15):  Reduction  of  cyclcstationary  processes-  to 

stationary  processes  via  phase-randomization 

Theorem  (2-16):  Equivalence  between  phase-randomizing  and  time¬ 
averaging  of  random  processes. 

Theorem  (2-17):  Preservation  of  system  cyclostationarity  by 
phase- randomi zat ion . 

Theorem  (2-18):  Reduction  of  cvclostationary  systems  to  widc- 

sensc  stationary  systems  via  phase-randomization. 

Theorem  (2-19):  Equivalence  between  phase-randomizing  and  time¬ 
averaging  random  systems. 

ii)  Generaliz'd  angle  modulation. 

Theorem  (2-20):  Preservation  of  cyclostationarity  by  a  stationary 
time- sea  1 c  trans forma ti on . 


Theorem  (2-21):  Generate, on  of  cyclostationary  processes  from 
periodic  signals  via  stationary  time-scale 
transformations. 

Model  (21):  Phase-modulated  signals. 

Model  (22):  Frequency-modulated  signals. 

Theorem  (2-22):  Generation  of  cyclostationary  processes  from 
PAM  time-scale  transformations. 

Model  (23):  Digital  phase-modulated  signals. 

Model  (24):  Digital  frequency-modulated  signals. 

Theorem  (2-23):  Generation  of  cyclostationary  processes  from 
stationary  processes  via  cyclostationai'y  time 
scale  transformations. 

Section  6:  Fandom  Mul ti -dimensional  Linear  Transformations. 

Theorem  (2-2»):  Generation  of  one-dimensional  cyclostationary 
processes  from  multi-dimensional  stationary 
processes  via  random  multi-dimensional 
eye  los  ta t i onary  sys terns . 

Section  7:  Nonlinear  Fandom  Transformations. 

a)  Zero-memory  random  nonl Lnoarities. 


Theorem  (2-26):  Preservation  of  cyclostat ionarity  of  random 
proces.es  by  random  jitter. 


Theorem  (2-27):  Preservation  of  cyclostationarity  of  random 
systems  by  random  jitter. 

In  concluding  this  summary  of  Chapter  II,  we  mention  two  interesting 
facts  which  arise  throughout  the  chapter:  1.  There  need  be  no  exact 
periodicity  in. the  realizations  of  a  cyclostationary  process:  The 
occurrence  of  pulses  in  jittered  PAM,  for  example,  is  not  periodic,  nor 
are  the  pulse-occurrences  in  a  Poisson  pulse-process  with  periodic  rate 
parameter.  The  periodicity  need  only  be  present  in  the  first  and  second 
order  ensemble  averages.  2.  The  mean  and  variance  for  a  cyclostationary 
process  can  be  constant  so  that  the  periodicity  in  these  quantities  is 
degenerate,  and  still  such  processes  can  exhibit  strong  periodic 
fluctuations  in  correlation,  us  exemplified  by  some  time-division- 
multiplexed  signals,  pulse-amp l itude -modulated  signals,  and  others. 

b)  Chapter  111:  Scries  Itcprcscn tut  ions  for  Cyclostationary 
Processes  (and  Their  Autocorrelation  Functions).  The  third  chapter  of 
this  dissertation  is  an  in-depth  treatment  of  scries  representations 
for  cyclostationary  (and  stationary)  processes  and  their  autocorrelation 
functions,  and  other  periodic  kernels;  and  includes  discussions  of  their 
application  to  the  solution  of  linear  ii.tcgral  equations  with  periodic 
kernels,  and  to  the  realization  of  periodically  time-varying  linear 
systems,  and  to  the  definition  of  a  generalized  Fourier  transform  for 
cyclostationary  processes. 

Three  types  of  series  representations  are  treated: 
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1.  Translation  series  representations  of  the  form 
M  “ 

x(t)  =  l  l  a  6  (t-nT)  He  (-«,») 
p=l  n=-«  ‘  * 

M 

kvft,s)  =  l  l  Apq  J n(t-nT)**(s-mT)  ¥  t.s  e  (-,»), 
xx  p,q=l  n,m=-“  R“m  p  q 


where  }  are  deterministic  basis  functions,  and  {{a  };  p  = 

P  np  r 

are  jointly  stationary  random  sequences  with  crosscorrelations 


E{a 


a*  }  =  Apq  . 
np  mq  n-m 


(T  is  the  period  of  cyclostationarity) . 

If  the  basis  functions  arc  doubly  orthonormal  in  the  sense  that 

/  i<  (t-nT)$*(t-mT)dt  =  65, 

J  1|)'1  vqv  '  pq  nnr 

then  the  representors  are  given  by  the  formulas: 


a  =  /  x(t+nT)4*(t)dt 

»P  .J  P 

to 

APq  =  //  k  (t)<J>  (t)dtdi 

n-m  xx  P  q 


Harmonic  series  representations  of  the  form: 


xtt)  .  I  » 

p  =  I 


V'  t  c  (-co,,u) 


k  (t.s)  =  l  k  (t  s)e*’”T,^pt*qs^1  V-  t,s  c 
xx  „*•  pq  v  ' 

p,q=-«  1  1 


where  the  {a(j}  are  jointly  stationary  processes  band limited  to 
[- 1/2T, 1/2TJ ,  and  with  crosscorrelations 
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E{a  (t)a*(s)>  =  k  (t-s). 
P  4  PH 


The  representors  are  given  by  the  formulas: 


a  (t)  =  /  w(t-r)x(-r)e  i27,PT/Tdx 

*  -OO 


where 


co 

k  (t-s)  =  //  w(t-T)w(s-Y)k  (T,v)e'j7l(pT'qY)/TdTdY 

PH  .co 


W(t)  A  Sin(wt/T) 


3.  Fourier  scries  representations  (for  autocorrelation  functions) 


of  the  form 


k  (t,s)  =  l  c  (t-s)^”^'1  V-  t,s  c  (-«,»), 


where  the  coefficient  functions  are  defined  to  be 


o  ff)  =  i  /  k  (t*T/2,t"/2)e"*i2lint//1dt. 
n  \  _.jy2  NX 

These  three  types  of  representations  are  developed  in  detail,  and 
the  relationships  existing  amongst  them  are  brought  to  light.  The  most 
general  representation  is  the  translation  series  representation,  since 
many  different  translation  series  representations  for  a  given  cyclostationary 
process  can  be  obtained  by  making  different  choices  for  the  basis 
functions.  In  fact,  a  generalization  of  the  Karhuncn-Loevc  representation 
is  shown  to  be  one  special  case  of  a  translation  scries  representation, 
and  the  Sampling  Theorem  provides  another.  It  should  be  emphasized 

that  these  translation-series  representations  are  countable  discrete 
representations,  and  are  valid  on  the  entire  real  line 
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The  specific  contents  of  this  chapter  are  outlined  below: 

Section  1 :  Introduction 

Section  2:  Translation  Series  Representations. 

a)  Definition 

Theorem  (3-1):  Existence  and  identification  of  translation 
series  representations. 

b)  Representation  in  terms  of  PAN!  processes. 

c )  Implementation  of  the  process-resolution  operation. 

d)  Examples. 

i)  Harmonic  translation  scries  representation. 

ii)  Walsh  translation  scries  representation. 

iii)  Karhunen- hoove  translation  series  representation. 

e)  Finite-order  translation  series  representations. 

i)  Band limited  cyclostationary  processes. 

ii)  Degenerate  cyclo.>tat ionary  processes. 

f)  Correlation  matrix  decomposition  and  solution  of  integral  equations. 

i)  Integral  Equations. 

Theorem  (3-2):  Explicit  solution  of  general  linear  integral 
equation  with  periodic  kernels. 

ii)  Correlation  matrix  decomposition. 

(1)  Infinite-dimensional  diagonal  matrices. 

Theorem  (3-3):  Existence  ,md  identification  of  infinite- 

dimensional  diagonal  correlation  matrices  for 
translation  series  repi i sent  at  ions. 


(la)  Random  video  signal. 

(lb)  Time-division-multiplexed  signal. 

(2)  Infinite-dimensional  diagonal -plus-finite-rank  matrices. 

Theorem  (3-4):  Matrix  inversion  formula. 

(2a)  Cyclostationary  processes  which  are  the  outputs  of 
periodically  time-varying  finite-dimensional  linear 
dynamical  systems  driven  by  white  noise. 

Theorem  (3-S):  existence  and  identification  of  diagonal -plus- 
finite-rank  correlation  matrix  for  translation 
scries  representation. 

(2b)  Timc-divis ion-multiplcxcd  process  with  component  signals 

which  have  been  generated  by  finite-dimensional  time- invariant 
systems. 

(3)  Finite-dimensional  diagonal  and  diagonal-plus-rank-1  matrices. 
(3a)  Timc-division-multiplcxcd  PAM  signals. 

(3b)  Synchronous  M-ary  signals. 

g)  Implementation  of  periodically  time-varying  linear  systems. 

h )  Generalized  Fourier  transform  for  cyclostationary  processes. 

Theorem  (3-6):  Equivalence  between  generalized  Fourier  transforms 

under  statistical  expectation  (extension  of 
Holly's  Theorem). 

Section  3:  Harmonic  Scries  Representation. 

a)  Definition. 

Theorem  (3-7):  Existence  and  identification  of  harmonic  series 
representation. 

b)  Resolution  and  reconstruction. 


3o6 


c)  Properties  of  the  harmonic  series  representations. 

(1)  Stationarity. 

(2)  Phase-randomization. 

(3)  Bandlimitedness. 

(4)  Time-invariant  filtering. 

d)  Examples. 

(1)  Amplitude-modulated  signals. 

(2)  Frequency-division-multiplexed  signals. 

(3)  Random  periodic  signals. 

e)  Solution  of  integral  equations  and  realization  of  periodically 
Time-varying  systems. 

Section  4:  Fourier  Series  Representation  for  Autocorrelation  Functions. 

a)  Definition. 

Theorem  (3-8):  Identification  of  Fourier  scries  coefficient 
functions  a.,  crosscorrclation  functions  for 
jointly  stationary  phase- randomized  processes. 

b)  Properties  of  the  Fourier  series  representation. 

i)  Phase-randomization  and  stationarity. 

ii)  Filtering  and  band  limit ing. 

iii )  Constraints  on  coefficient  functions. 

c)  Solut ion  of  integral  equations  and  realization  of  periodically 
time-varying  systems . 

We  conclude  this  summary  of  Chapter  Ill  by  pointing  out  the  fact 
that  the  essence  of  the  representations  presented  here  is  that  they 
effect  a  decomposition  of  a  single  cyclostationary  process  into  a  multiplicity 
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of  jointly  stationary  components.  Such  representations  enable  one  to 
convert  many  single-variable  problems  (involving  cyclostationary  processes) 
with  periodically  varying  parameters  to  multi-variable  problems  with 
constant  parameters.  This  is  the  case  for  optimum  linear  filtering 
problems  as  illustrated  in  Chapter  IV. 

Similarly,  the  autocorrelation-function  representations  presented 
here,  when  applied  to  the  representation  of  impulse-response  functions, 
effect  a  decomposition  of  a  single  periodically  time-varying  linear 
system  into  a  multiplicity  of  time- invariant  filters  and  periodic  modulators. 

c )  Chapter  IV:  Least-Mcan-Squared-Hrror  Linear  Estimation  of 
Cyclostationary  Processes.  In  this  fourth  chapter  we  consider  a 
particular  application  of  the  theory  developed  in  the  preceding  two 
chapters.  These  earlier  chapters  serve  mainly  as  a  theoretical  base  upon 
which  various  problems  dealing  with  random  periodic  phenomena  can  be 
formulated  and  solved.  Chapter  II  provides  a  base  for  obtaining 
mathematical  models  for  physical  systems  and  signals,  so  that  physical 
problems  can  be  formulated  mathematically,  and  Chapter  111  provides  a 
base  for  simplifying  the  mathematical  models  so  that  analyses  can  be 
simplified,  and  solutions  obtained. 

Although  the  application  in  this  fourth  chapter  is  but  one  of  many 
possibilities,  it  has  special  significance  in  that  it  has  been  the 
primary  motivation  for  developing  the  theory,  furthermore,  the  problem 
considered  in  this  application  is  of  significant  practical  (as  well  as 
theoretical )  value  in  that  Us  solution  provides  means  for  improving  the 
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quality  of  present  communication  systems  (and  other  signal  processing 
systems),  and  a  means  for  evaluating  certain  performance  measures  for 
newly  proposed  (as  well  as  existing)  communication  systems,  and  it  leads 
to  useful  insight  into  the  nature  of  optimum  signal  pi'ocessing. 

Specifically,  the  problem  considered  is  that  of  least-mean-squared- 
error  linear  estimation  of  imperfectly  observed  cyclostationat'y  processes-- 
or,  in  communications  jargon,  optimum  filtering  of  noisy  distorted 
cyclostationary  signals.  The  particular  aspects  of  the  optimum  filtering 
problem  that  are  investigated  in  this  chapter  are  the  structure  and  the 
performance  of  optimum  linear  filters  for  cyclostationai'y  signals.  In 
fact,  we  prove  (and  demonstrate  with  numerous  examples)  the  proposition 
that  optimum  filters  for  cyclostationary  signals  are  periodically  time- 
varying  systems  which  can  yield  significantly  improved  performance  over 
optimum  filters  which  are  constrained  to  be  time-invariant  and  which  thereby 
ignore  the  cyclic  character  of  the  cyclostationary  signals.  We  also  show 
that  optimum  linear  receivers  consist  of  demultiplexor-demodulators 
followed  by  optimum  time- invariant  fiitcrs--thc  order  of  these  two 
operations  being  the  reverse  of  that  sometimes  encountered  in 
communication  systems.  These  results  are  of  considerable  importance  in 
view  of  the  great  preponderance  of  eyclostat ionary  signals  in  communication 
systems  (as  emphasized  throughout  Chapter  II).  Specific  examples  which  are 
worked  out  in  detail  arc  included  in  the  outline  below: 

Section  1:  Introduction 

Section  2:  The  Orthogonality  Condition. 
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Theorem  (4-1):  Lemma  for  the  application  of  the  Projection 
Theorem  to  the  optimum  filtering  problem. 

Theorem  (4-2):  Statement  of  the  orthogonality  condition  as 
it  applies  to  the  optimum  filtering  problem. 

(1)  "Infinite-dimensional"  filters. 

(2)  "Finite-dimensional"  filters. 

Section  3:  Optimum  Time- Invariant  Filters  for  Stationary  Processes. 

Theorem  (4-3):  Bounds  on  mean-squared  estimation-error. 

Section  4:  Optimum  Time- Invariant  Filters  for  Cyclostationary  Processes. 

Theorem  (4-4):  Equivalence  between  the  optimum  filters  which 

minimize  the  time-averaged  error  for  cyclostationary 
processes,  and  those  which  minimize  the  constant 
error  for  the  phase -randomized  versions  of  the 
cyclostationary  processes. 

Section  5:  Optimum  Time-Varying  Filters  for  Cyclostationary  Processes. 

a)  Introduction. 

b)  Solutions  based  on  translation  series  representations. 

Theorem  (4-5):  General  solution  for  the  impulse-response  function 
for  the  optimum  time-varying  filter  for  cyclo¬ 
stationary  processes,  and  solution  for  the 
minimum  estimation-error. 

Theorem  (4-6):  Extension  of  Theorem  (4-5). 

Theorem  (4-7):  Special  case  of  Theorems  (4-5),  (4-6). 


Theorem  (4-8):  Bounds  on  mean-squared  estimation-error. 


i)  Estimation  of  cyclostationary  processes  on  an  infinite  interval. 

Theorem  (4-9):  General  solution  for  the  impulse-response 

function  for  the  optimum  time-varying  filter 
for  cyclostationary  processes,  and  solution  for 
the  minimum  estimation-error. 

ii)  Estimation  of  periodic  processes  on  a  finite  interval. 

Theorem  (4-10):  Solution  for  the  impulse-response  function  for 

the  optimum  time-varying  smoother  for  periodic 
processes  in  additive  white  noise,  and  solution 
for  the  minimum  estimation-error. 

Theorem  (4-11):  Counterpart  of  Theorem  (4-10)  for  the  optimum 
time- i nva ri a n t  smoother . 

d)  Solutions  based  on  the  Rnirier  series  representation  for 

correlation  functions. 

Theorem  (4-12):  Conversion  of  integral  equation  for  the  impulse- 
response  function  for  the  optimum  time-varying 
filter  to  an  infinite  set  of  linear  algebraic, 
equat ions . 

Theorem  (4-13):  Specific  case  of  Theorem  (4-12)  which  yields  an 
explicit  solution. 

c)  Solutions  based  on  inverse-operator  and  feedback-system  methods. 

i)  Introduction. 

ii)  Examples. 

(1)  lv’SS  signal  subjected  to  time-invariant  dispersion  and 
periodic  attenuation,  and  additive  white  noise. 


! 


I 


i 


541 

■  .1  ■ 

•  ’  5  •,  i 

(2)  CS  AM  signal  in  additive  WSS  white  noise.  < 

i  i 

(3)  WSS  signal  in  additive  CS  white  noise. 

1  j 

(4)  CS  signal  (obtained  from  periodically  time-scaled  KSS  1 

!  I  * 

process)  in  additiye  WSS  white  noise.  ' 

* 

Section  6:-  Examples  of  Optimum  filters  and  improvements  in  Performance. 

a )  Examples  employing  translation  series  representations. 

t 

i)  Prequency-shift-keycd  signals.  ! 

r 

ii)  Time-division-multiplcxcd  PAM  signal's. 

•  1  I  1  , 

iii)  Prequency-division-multiplexed  signals. 

'i  ,  .  i 

iv)  Pulsc-ampli„tude-modulated  signals.  .  i 

;  i 

v)  PAM  signals  through  random  channels. 

vi)  The  video  signal.  i  , 

’vi i)  Tiinb-division-multiplcxed  signals.!  ! 

•  i 

b)  Examples  employing  the  harmonic  series  representation, 

ii)  Preqtiency-divi  sion-multiplcxcd  signals.  ,  1 

:  .  i 

ii)  Amplitude-modulated  signals. 

i  i  i  i 

iii)  The  random  sinusoid. 

t 

c)  Examples  employing  the  I'ourier  scries  representation. 

i)  Amplitude-modulated  signals.  1 

i  i  . 

ii)  Bandpass  PAM  signals.  1  , 

j  , 

iii)  Improvement  for  general  PAM. 

I 

i 

d)  Examples  employing  the  inverse-operator- feedback-system  method. 

i)  .Periodically  attenuated  KSS  signal  in  KSS  noise.  i 

ii)  AM-type  signal  in  WSS  noise. 

iii)  WSS  signal  in  white  CS  noise. 

i  •  i  ,  ‘ 

» 

,,  iv)  Doppler-shifted  Poisson  pulse-ttaih. 
v)  Time-divi'S ion-mul  t  iplexed  signals. 


I 
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f)  Summary. 

Theorem  (4-14):  Demultiplexing  and  demodulation  precedes  time- 
invariant  filtering  in  optimum  linear  receiver. 


We  conclude  this  summary  of  Chapter  IV  by  mentioning  the  intuitively 
pleasing  result  that  optimum  time-varying  filters  for  cyclostationary 
processes  can  always  be  decomposed  into  a  sequence  of  three  signal 
processing  operations:  1.  The  received  cyclostationary  process  is 
decomposed  into  one  or  more  (jointly)  stationary  processes.  2.  The 
stationary  component-processes  are  passed  through  time- invariant  filters 
(often  Wiener  filters).  3.  The  filtered  components  are  recombined  to 
form  the  optimally  filtered  cyclostationary  process.  In  most  of  the 
examples  presented,  this  sequence  of  operations  takes  one  of  the 
following  specific  forms:  demodulator- filter-modulator,  demultiplexor- 
filter-multiplexor,  inverse  time-scale  transformation— filter--time-scale 
transformation,  l-'or  those  particular  cyclostationary  processes  which 
decompose  into  a  single  stationary  process,  the  three  signal  processing 
operations  arc  more  accurately  described  as:  i.  Removal  of  periodic 
fluctuations.  2.  Time- invariant  filtering.  3.  Re-insertion  of  periodic 
fluctuations. 

Another  result  worth  emphasizing  is  'hat  cvclostationary  processes 
whose  mean  and  variance  are  constant  can  still  possess  a  high  degree  of 
cyclostationari ty  in  the  sense  that  optimum  time-varying  filtering  can 
yield  substantially  lower  mean -squat cd  error  than  optimum  time- invariant 
filtering. 


Finally,  we  mention  that  optimum  time-varying  filters  for 
cyclostationary  processes  appear  to  yield  the  greatest  reduction  in 
mean-squared  error  (below  that  resulting  from  optimum  time-invariant 
filters)  when  the  signal-to-noise  ratio  is  high,  but  can  yield 
substantial  reductions  even  at  signal-to-noise  ratios  as  low  as  unity. 
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2.  Suggested  Topics  for  Further  Research 

There  are  various  topics  dealing  with  cyclostationary  processes 
which  deserve  fuvther  research.  We  outline  below  several  such  areas  of 
investigation. 

a)  Degree  of  cyclostationarity.  The  only  effective  means  we 
have  for  determining  the  filtering  advantage  to  be  gained  from  a  periodically 
time-varying  filter  operated  in  synchronism  with  the  received  cyclo- 
stationary  signal  is  to  carry  out  the  complete  solutions  for  the  optimum 
time-varying  and  time- invariant  filters,  and  to  evaluate  the  performance 
functionals.  It  would  therefore  be  desirable  to  devise  a  functional 
which  will  indicate  the  "degree  of  cyclostationarity"  of  a  process  with 
regard  to  filtering-performance  improvement  that  inherently  can  be 
attained.  Obviously,  a  "highly  cyclostationary"  process  will  exhibit 
the  largest  improvement.  One  might  expect  that  a  constant-variance 
cyclostationary  process  must  have  a  limited  degree  of  cyclostationarity 
The  video  signal,  for  example  has  constant  variance  and  for  typical 
parameter  values  the  performance  improvement  factor  is  essentially 
negligible.  However,  the  PAM  signal  with  full  duty-cycle  rectangular 
pulses  has  constant  variance,  >et  has  an  unlimited  improvement  factor 
for  low-noise  conditions.  One  possible  candidate  for  measuring  the  degree 
of  cyclostationarity  is  the  relativi  siie  of  the  cff-Jiagonal  terms  in  the 
HSR  correlation  matrix.  For  example,  if  the  off-diagonal  terms  vanish 
then  the  process  is  stationary,  and  by  definition  there  is  no  improvement. 

A  relatively  simple  functional  for  indicating  degree  of  cyclostationarity 
or  obtaining  bounds  on  improvement  hould  find  extensive  application  in 
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further  studies  since  certain  aspects  of  performance  can  be  determined 
without  having  to  carry  out  a  complete  solution  to  the  estimation  problem. 

b)  Implementation  of  periodically  time-varying  filters.  Although 
most  of  the  examples  of  optimum  filters  presented  in  Chapter  IV  wore 
finite  structures,  the  general  solutions  provided  in  Theorems  (4-5, 6, 7, 9) 
are  based  on  infinite  series  representations,  and  result  in  infinite 
structures  (infinite  number  of  pai*allcl  paths.)  As  discussed  in  Chapter  III, 
truncated  versions  of  the  representations  may  provide  satisfactory 
approximations  and.  finite  structures,  and  the  implementation  of  such 
structures  can  be  enhanced  by  the  use  of  convenient  basis  functions  which 
result  in  the  replacement  of  modulators  by  switches,  and  by  the  use  of 
digital  filters  to  implement  sampled-data  filters.  However,  these  filters 
probably  do  not  have  the  minimum  complexity  for  a  given  degree  of 
approximation  to  the  optimum  filter.  Less  complex  filter  structures 
might  likely  result  from  an  a-priori  choice  of  basis  filters,  and 
solution  for  the  corresponding  optimum  "finite-dimensional  filter"  as 
discussed  in  Section  2  of  Chapter  IV.  In  fact,  the  basis  filters  can 
be  chosen  for  adjustability  so  that  changes  in  signal  and  noise  parameters 
can  be  conveniently  accomodated.  An  investigation  into  the  trade-off's 
available  between  convenience -of- implementation  and  performance  could 
produce  some  useful  results. 

e)  Synchronizing  and  tracking.  A  random  signal  can  be  cyclostationary 
even  though  there  may  be  no  strict  periodicity  in  the  occurrence  of  the 
random  events  comprising  the  signal,  for  example,  in  direct  contrast  to 
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the  periodic  occurrence  of  pulses  in  a  synchronous  PAM  signal,  the  pulse- 
occurrences  in  a  Poisson  pulse-train  process  with  periodic  rate  parameter 
are  random- -only  the  average  rate-of-occurrences  is  periodic.  In  fact 
some  signal  processes  exhibit  cyclic  fluctuations  but  are  only  "quasi¬ 
cyclostationary"  since  the  period  of  the  cyclic  fluctuations  varies  slowly 
with  time.  Obtaining  knowledge  of  the  phase  of  such  non- synchronous 
cyclostationary  (and  quasi-cyelostationary)  signals  is  not  as  straight¬ 
forward  as  it  is  for  highly  structured  synchronous  cyclostationary  signals 
such  as  PAM.  Thus  it  would  be  desirable  to  develop  methods  for  tracking 
cyclostationary  signals  in  order  to  extract  useful  timing  information  for 
optimum  filtering,  and  other  signal  processing  operations.  The  phase 
lock  loop  and  some  of  its  variations  such  as  the  Costas  filter  loop  and 
the  squaring  loop  have  found  much  application  for  symbol  synchronisation 
for  synchronous  pulse-trains,  and  for  carrier  synchronization  [40].  One 
approach  to  the  timing  recovery  problem  for  more  general  cyclostationary 
signals  then  would  be  to  extend  these  and  related  ideas  which  have  been 
successful  for  the  more  structured  signals  [1,40].  future  investigations 
might  sock  to  evaluate  the  off*  <.ts  ol  trade-offs  in  complexity  between 
the  tt me- varying  filter  and  the  tracking -loop  structures  on  overall 
performance. 

dj  Model  ling.  A  cont  inued  study  of  appropriate  models  for  random 
processes  resulting  from  various  l> pes  of  sampling,  scanning,  and  multi¬ 
plexing  open  lions  is  essential  in  order  to  take  full  advantage  of  the 
results  in  tins  thesis.  In  p.i’liculai  ,  one  might  concentrate  on  operations 


involving  random  parameters,  such  as  scanning  speed  or  clocking  rate, 
which  produce  non-synchronous  signals.  The  resulting  processes  may  be 
either  cyclostationary  or  "quasi-cyclostationary",  as  mentioned  previously, 
depending  on  the  models  chosen.  See,  for  example,  Section  7  of  Chapter  U 
on  jitter. 

e)  Signal  Design.  Related  to  the  modelling  problem  above  are 
questions  concerning  the  design  of  the  sampling,  scanning,  and  multiplexing 
operations  with  regard  to  introducing  a  sufficient  degree  of  cyclostationarity 
for  satisfactory  receiver  operation,  with  respect  to  both  timing 
extraction  and  filter  performance.  An  Investigation  of  such  ideas  could 
provide  the  basis  for  the  joint  optimum  design  of  transmitter  and  receiver. 
Previous  studies  have  treated  an  aspect  of  this  known  as  the  power 
allocation  problem  [40]  wherein  the  trade-off  between  power  in 
synchronizing  signals  and  in  informution-bcaring  signals  is  examined.  A 
new  approach  to  this  problem  might  lie  based  on  degree  of  cyclostationarity. 
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